Minutes of Detector Characterization Teleconference
(September 6, 2002)
Present:
Caltech: Zweizig
Hobart-Smith: Penn
LHO: D'Ambrosio, Drever,
Gustafson, Ito, Landry, Lazzarini, Raab, Rahkola, Schofield, Sigg
LLO: Chickarmane, Frolov,
Gonzalez, Marka
MIT: Cadonati, Katsavounidis
Michigan: Riles
Oregon: Leonor
Syracuse: Saulson
S1 Roundup: (S1
Web Page )
On Tuesday Dick noticed a "two-thump heartbeat" in H1 with a frequency
of 88/min, leading to a comb of 1.47 Hz lines up to about 150 Hz (can go
higher if filtering is disabled). This persisted until a small earthquake
broke lock and required realignment. Afterward the two thumps became four
thumps. The source is unknown. (Note added next day: the owl crew last
night tracked down the heartbeat to undamped side motion of the end-X mass
modulating optical gain; the motion was originally set off by an Oregon
earthquake and has been fixed by turning the servo gain down temporarily
and then back up. See elog
entry.)
At Livingston, S1 has been coincident with intensive and nearby
logging during the day, even on Labor Day, leading to a science mode livetime
of about 42%. But at night conditions have been reasonably good. Frequent
realignments are required, but otherwise running has become routine, and
the L1 spectrum has remained stationary. Reported glitch rates online are
low, as for H1.
-
Lock Statistics
(G.Gonzalez)
In addition to the single-IFO statistics being produced daily by scimons,
Gaby has been using a matlab program looking at conlog output to find 2-
and 3-IFO coincidence science mode segments and to calculate coincidence
livetimes. As of the morning of September 6, the livetimes were L1-H1 (30%),
L1-H2 (34%), H1-H2 (46%), and L1-H1-H2 (24%). The last number can (almost)
be compared to 11% for E7, but Gaby has not yet enforced the condition
on coincidence segments of being at least 5 minutes long (as is done for
single-IFO livetime calculations).
-
Data Quality (J.Zweizig)
John went over an outline
of the Data Quality investigation team plans for producing a more refined
set of high-quality database segments. A dedicated DMT monitor called DataQual
(based on John's versatile PSLMON programs like RMSBands and ZGlitch) is
creating trends of band-limited rms power for four channels per IFO in
nearly 40 bands and trends of glitch rates with various thresholds / filtering
for about ten channels per IFO. Rauha and Peter Saulson have been looking
at selected DataQual power bands and glitch rates for the burst upper limit
group and provided sample plots: Rauha(1
23
4); Peter (1
2). John plans to revise the glitch
finder to notch out some prominent lines that tend to mask glitches by
their non-Gaussian contribution to total RMS (used for thresholding). Looking
at trends offline away from the sites is problematic if one doesn't have
a local copy of the strictly Sun-platform data viewer. Even there John
reports some occasional hiccups in getting data from the network data server.
(KR once again pleaded for a volunteer to write a general-purpose
trend viewer for the LSC, one that operates on a linux or generic unix
platform. Peter Shawhan first publicized the need for such a viewer two
years ago, but no volunteer has yet stepped forward. This would be a very
nice, immediately useful tool for a graduate student or postdoc to create
for all of us. Interested parties: please see KR.).
Based on what is found in looking at trends, criteria will be set for
defining new database segments more restrictive than the nominal "science
mode" used in present insertions. If there is a demand, John could also
produce customized, more restrictive (or less restrictive) segments for
particular analysis groups.Aside from the information provided by DataQual
and other DMT monitors, there is also information from the offline DSO's,
in particular the burst group DSO's on bottom-line glitch rates. Laura
maintains a web
page at MIT with compiled results of the burst DSO's (including history
plots of rates vs times) which should be quite useful.
-
Calibration Status (M.Landry)
Calibrations were carried out on H2 and L1 the first evening of the
run. Calibrations for H1 had to wait a couple of days because of initial
locking troubles. Since then there have been autocalibrations twice per
day by operators, the results of which are available on the calibration
web page and periodically entered into the elogs at both sites. Click
here to see an early comparison by
Rana of the three IFO strain curves. Calibration lines have been injected
at about 37 Hz and 973 Hz on all three IFO's during the run. There was
some confusion initially about the strength of the 973 Hz line, since LineMonitor
was having trouble detecting a stable signal strength, but the problem
turned out to be in the tracking, not in the line itself, and has now been
fixed by Sergey. The last calibration run is planned nominally for Sunday
night. Fred suggested that if we were very close but not quite at 100 hours
of triple-coincidence data at that time, it would be tempting to postpone
the calibration in order to reach the round-number milestone. On Monday
night the hardware / software freeze will remain in effect to permit astrophysical
injections for the burst, inspiral and stochastic upper limits groups.
If necessary/desirable, calibrations could be done the same night.
At the moment, the EPICS alarm that reports the absence of a calibration
line from an excitation test point fails to detect an occasional failure
mode in which the operator forgets to turn the excitation back on after
an autocalibration. Mike suggested it would be more reliable to write a
DMT monitor that sets an alarm if the AS_Q power in the expected very narrow
band falls below a certain level. There was some discussion of correcting
for drifting calibrations due to misalignment. A simple frequency-independent
scaling factor based on the strength of one calibration line is not suffficient
(in fact, the 37 Hz line has been observed to drift up while 973 Hz falls
as alignment degrades). The optical gain contribution to the total closed-loop
gain is not flat. Gaby mentioned that she is writing up a note on this
subject for the benefit of the upper limits groups. The calibration team
hasn't yet confronted this problem, but will be looking at it when time
permits after the run. It hasn't been decided yet in what form to provide
time-dependent transfer functions.
-
Reduced Data Set Generation (I.Leonor)
The RDS generation has run smoothly since the 2nd day of S1.
The LDAS CreateRDS command has been used for the generation and seems to
work fine. One complication, however, is that version 5 frames are being
produced, which cannot be read by any non-LDAS program at the moment. John
mentioned that he is trying to link the latest version-5-compatible framecpp
library to the DMT and having technical problems that he believes are not
fundamental. Once he gets this working, John expects to have two different
DMT versions available, one to read/write frame 4 (current standard) and
one to read/write frame 5 (LDAS special). Albert reported that it will
be a month before version 6 frames will be produced for S1. In principle,
programs that link to the latest frame library from Benoit should be able
to read those version 6 files. Some online programs (e.g,. the DTT) will
not immediately be able to read them, however.
Isabel was asked by Laura to provide to the burst upper limits group
a highly stripped-down RDS in frame version 4 of the triple-coincidence
playground data (~3 hours) that Gaby and Peter Saulson recently designated.
Once that playground set is defined for the whole run, it should amount
to about 10 hours of data. Peter agreed to KR's request to query the other
upper limits group chairs to see if a coordinated mini-RDS should be created
- on condition that Peter be allowed to specify a short deadline for reply.
-
S1 Roundtable Discussion (All - Procedures, Investigations, Software problems)
KR solicited suggestions for improving scimon shift procedures
and better supporting investigations, along with reports of major software
troubles. Summary of issues that were raised:
-
Scimons should read e-log entries! (KR agreed to update the scimon guidelines
with this explicit directive.)
-
More standardized templates are needed for generating status plots (e.g.,
Rauha's day-by-day DataQual summaries) so that less time is spent creating
them and more time studying them.
-
Currently outstanding problems should be noted on a whiteboard in the control
room so that not every shift transition requires reporting the same 3-day
old problem.
-
Keeping two IFO's running in good condition at Hanford tended to keep the
expert scimon plenty busy, leaving little time to work on the scimon checklist.
It might be appropriate to keep two scimons on duty at LHO even when we
enter a steady state of one scimon at LLO.
-
More systematic and centralized archiving of reference spectra would be
useful so that one can easily compare spectra taken at different times.
Fred and KR reminded everyone that Tim Olson and his students at Salish-Kootenai
College are working on this very project, using a DMT monitor to record
spectra and the DTT for retrieving them.
-
Peter remarked that the same systematic archiving and retrieval would be
desirable for other measures of IFO performance, such as trends of glitch
rates.
-
Fred raised the issue that we now take a hands-in-pocket approach during
data runs. Normally, if the IFO is running reasonably well, we leave it
alone, even if alignment is drifting significantly. He wondered if we should
instead spend some time each day measuring the parameters of the machine
(e.g., closed-loop gains for four main degrees of freedom) so that we avoid
major drifts away from the optimum, drifts that lead to difficult lock
reacquisition (c.f. problems above with H1 early in S1). A similar issue
arises in relieving the DC offsets imposed by the wave front sensing system.
The offset corrections are done by hand now, but an automatic script that
adjusts biases to agree with WFS values would be preferable.
-
Steve reported a chronic problem in using ssh and root graphics together,
leading to segmentation faults in the BicoMon monitor. The problem can
be avoided by setting xwindows display parameters directly, bypassing the
ssh encryption, but Steve wondered if there were a systematic problem with
ssh tunneling in the control room. KR and others reported flakiness in
tunnel connections between the LHO and LLO control rooms that interfered
with displaying LHO and LLO DMT viewer plots simultaneously (e.g., LockLoss
status). Daniel ascribed the problem to multiple security firewalls getting
in the way.
DMT Software Status Review:
-
DMT infrastructure - J. Zweizig
No update - John had to leave early. But see comments above
about framecpp and version 5 frames.
-
Global summary page(s) - S. Marka
One of the major components of the web-based global status summary
page Szabi is working on is a display of up-to-date calibrated spectra
from the three interferometers. He now has a prototype
of that component working, using the spectra produced periodically by Patrick
Sutton's SenseMonitor. At the moment, only the spectrum in the 300-1400
Hz range (used by SenseMonitor for calculating binary inspiral range in
kpc) is displayed, and there is no correction for drifts in transfer function.
Patrick is working on a post-S1 version that will address both issues.
Updates from DMT monitor authors (updates since August 9 meeting):
-
Dave Chin: LockLoss and ServoMon (KR reporting)
There has been some confusion about one entry in the LockLoss livetime
summary pages. The total number of hours in lock since the monitor was
started is provided, but the wording makes it sound as if the current lock
has lasted that long. The wording will be fixed after S1 and a separate
entry added with the number of seconds of the current lock. Daniel requested
that a table be added showing the lengths of the last ten locks. There
is also a new problem with the reading of LockLoss trends which Dave is
investigating.
.
-
Ed Daw: blrms and PeakMon (no update)
-
Masahiro Ito: glitchMon
GlitchMon has been running smoothly since the start of S1. No known
bugs.
-
Sergey Klimenko: LineMonitor and WaveMon (no update)
-
Szabi Marka: IRIG-B and TimeMon
A bug in the TimeMon code was fixed by Daniel early in S1. There have
been some real timing problems reported at LLO recently, requiring cpu
reboots. The timing investigation team will be determining at both sites
(for the first time) the time offset correction needed to find the correct
arrival time of a gravitational wave signal.
-
Adrian Ottewill: MTLineMon and CorrMon (no update)
-
Steve Penn: BicoMon
The GUI interface now informs the user in advance of the time required
to compute bicoherence for a given set of requested frequency resolution
and time interval. For those wanting mHz resolution, it may be advisable
to look back in time at data on disk instead of waiting for real-time data
to flow in. Steve has identified some bilinear pathologies which he will
be posting on the results page of the bilinear couplings investigation.
He is also looking at ways to further speed up his code, including heterodyning
techniques when a user cares only about a limited frequency band.
-
Rauha Rahkola: absGlitch and eqMon
Now that glitchMon has absolute thresholding, the need for absGlitch
is not so clear. Rauha urged people to try using glitchMon for absolute
thresholding, since it supports more DMT output features and since Masahiro
keeps it up-to-date.
-
Daniel Sigg: MultiVolt (no update)
-
Patrick Sutton: SenseMonitor and RayleighMonitor (no update)
-
Natalia Zotov: PTMon (report by e-mail)
Natalia has fixed the trouble she was having in communication between
her C++ DMT code and the USGS C code she interfaces to. Her next task is
to implement triggers with reasonable threshold settings.
-
John Zweizig: BitTest, HistCompr, PSLMon , and SegGener
(no
update but see DataQual discussion above)
A.O.B
-
Next meeting: October 4 telecon at 1:30 p.m. EDT