Present:

     Caltech: Ron Drever,  Peter Shawhan, Patrick Sutton, Stan Whitcomb, John Zweizig
     LHO: Mike Landry, Luca Matone
     LLO: Brian O'Reilly
     Michigan: Keith Riles
     MIT: Laura Cadonati, Erik Katsavounidis
     Oregon: Isabel Leonor, Rauha Rahkola, Brian Stubbs
     Penn State: Mike Ashley
     Salish-Kootenai: Tim Olson
     Syracuse: Alexander Ageev, Peter Saulson

S2 Reports ( S2 web page )

• Update on S2 Data Quality segments (K. Riles)

Version 2 of the S2 data quality segments have been released that reflect quantitative understanding to date of bad data stretches. For various reasons, the L1 IFO has received more attention than H1 and H2. Quite a bit needs to be addressed in version 3, including thresholds on DataQual glitch rates and band-limited RMS, along with periods of unreliable calibration and timing.

S3 Planning

• Nominal schedule: Friday October 31, 2003 - Monday January 5, 2004 (66 days) - Extension possible if pre-isolation equipment not ready for LLO installation in January. A week-long engineering run will be started two weeks before S3 to see where we stand, with an intervening week for making minor repairs / corrections before science running begins. Stan mentioned his goal of entering a phase of commissioning where the IFO's will be left in lockable states with data acquisition recording when possible. Two weekend "mini-runs" will be planned in the weeks leading up to the engineering run, probably both in September. PeterSa and KR urged that DMT and DTT figures of merit be displayed routinely in the control room, as soon as commissioning permits.

 
• Scimon shifts: The nominal plan is to continue as in S2. KR will allocate blocks to group contacts who will then provide names for individual slots. Shifts will be 8 hours long and manned 24/7 at both sites. More experimentation with manning shifts remotely from the MIT and Caltech control rooms will be tried. KR has requested at least one month's advance notice of run extension beyond January 5

 
• Operational changes from S2: KR presented a list of suggestions for improving data run operations, based on previous meetings, emails and discussions, which led to a roundtable discussion, including new suggestions:

 
• Formalize investigation/search activities for "quiet" scimon shift times - To make better use of "quiet time" when the IFO's are performing stably, all scimons will be asked to join at least one astrophysical search group or S3 investigation team and carry out investigative tasks when conditions permit. An expanded list of open topics for investigation will also be posted.

 
• Automation of plot creation / storage without deterring examination - Creation of daily summary plots is mildy cumbersome, but worse, seems highly subject to human error in getting start/stop times, filenames, etc correct. A number of scimon screwups in S2 required tedious reconstruction of plots. KR will talk to Rolf Bork and others about an automated creation of dataviewer trends. It was agreed by all that automated logging of plots leads to the danger that no one in the control room looks closely at the figures. PeterSa suggested that scimons be required to add annotations in the elog to the automatically generated plots.

 
• Automated production of combined RDS for inter-site correlation studies - Robert needs a combined reduced data set of certain PEM channels in order to study inter-site environmental correlations. LDAS now supports the infrastructure for merging data from the two sites at Caltech. Isabel volunteered to take responsibility in S3 for ensuring that combined RDS creation, using the infrastructure that Phil Charlton has recently put together. It's expected that there would be a latency of about a week between data taking and this RDS creation.

 
• Combine calibration curves and drift corrections into single file - It would be desirable for SenseMon to produce frame files that combine together both the alpha/beta minute trends and the reference curves for which those coefficients apply, to avoid any possibiity of downstream errors in interpretation. This worry is especially acute when SenseMon is rerun later to determine more accurate coefficients for certain data intervals. MikeA (who is taking over responsibility for SenseMon) volunteered to work on splitting SenseMon into a calibration-specific tool ("CalMon") for use by many users and which would generate the combined frame file and a separate piece for computing inspiral range.  John remarked that it would be desirable for SenseMon/CalMon to detect automatically from appropriate EPICS channels when a relevant gain change has been made. Patrick and Mike confirmed that the most likely culprit (DARM gain) will be monitored in the next version of the program(s) so that users need no longer change the config file and restart the monitor each time that gain changes.

 
• Setting reference states for 1-button conlog comparisons - A request has been made for a conlog option allowing a 1-button-click comparison of all current relevant IFO control settings with a reference set, where it would be possible to redefine the reference set, as needed. A similar feature is already supported by an IFO-status MEDM page, where one must manually enter the reference settings and the variables to be monitored. After some discussion, it was decided that conlog is complementary enough in its comprehensiveness and flexibility that the 1-button feature would be worthwhile. PeterSh volunteered to implement it.

 
• More attention given to describing data quality of segments day-by-day - It would be desirable to define standard good-science segments in near real-time so that scimons/operators can annotate the segments with relevant comments that affect judgement of data quality. Our automated monitoring tools do not capture all that may later affect astrophysical analysis. Much discussion ensued. Erik suggested that an automated system be set up with a maximum segment length and for which various quantities would be computed and logged, ideally in the form of a web page for each segment. [The present S2 segment definition allow segment lengths to range from 1 minute to as long as the length of a lock stretch (66 hours maximum in S2)]. KR worried about details in practical implementation of automated logging combined with the elog and with manual annotations. KR will consult with various experts and see what  can be implemented for S3.

 
• More wall & CRT displays with figures of merit - KR felt strongly that more figures of merit would have been valuable to display routinely on the control room walls in S2. As it was, some figures had to be displayed on ad hoc workstation monitors, reducing their visibility and often leading to their replacement by other workstation displays. Both sites seem to have potentially more room for display. KR will speak with Fred and MikeZ to see what can be added by S3. PeterSh remarked that in some cases in S2, figures of merit could have been displayed more compactly.

 
• Lower-threshold communication between the sites (e.g., computer chat room) - Albert had commented in the past that he felt communication between the site control rooms had been less than desirable because operators or scimons were reluctant to phone the other site unless there were an "important" issue. He suggested implementing an online computer chat program and noted that the GLAST collaboration uses a commercial AOL program. In the ensuing discussion here, no one spoke strongly in favor of a chat room, citing 1) the belief that the communication threshold is already quite low and that 2) a chat room gives no guarantee that a sent message will be immediately read on the other end. But the suggestion was made that there be regularly scheduled calls between the sites to ensure some communication (to reduce dependence on the gregrariousness of particular scimons). In a post-telecon e-mail, MikeA also noted that a chat room provides a lower threshold way to query the other site for its status, not just to report one's own, which is especially useful when coincidence running issues arise. Based on the discussion, KR is inclined to continue relying upon ordinary phones, but to encourage explicitly in scimon instructions an attitude of "When in doubt, call the other control room," with perhaps regular scheduling of 5-minute status-check calls.

 
• DMT improvements:

 
• More info from SenseMon (integrand, cumulative integral, other masses) - Patrick and MikeA confirmed that the next version of SenseMon will provide more information about the computation of the inspiral NS/NS range, including a plot of the integrand which is summed to obtain the range, and a plot of the cumulative integral from low frequency to high. It is hoped these curves will give more guidance on which noise sources are hurting us most. SenseMon will also provide inspiral range for a BH/BH system of 10 solar masses each.

 
• Background trending from BicoMon - Neither SteveP nor Vijay was present, but at previous meetings they have promised to have continuous background monitoring of known bilinear artifacts before S3.

 
• Dedicated suspensions monitor - BrianS is working on a dedicated monitor for suspensions (Rauha had put together a quick n dirty version for S2) to warn of out-of-tolerance coil and photodiode currents, with an eye to avoiding the side-motion-induced "heartbeat" seen for a few days during the S1 run.

 
• More flexible alarm handling (e.g., silencing with timeout) - KR and many others felt that temporary DMT alarm silencing by the operator/scimon would have been desirable during S2, for a variety of situations. KR will talk to Szabi to see what can be done for S3.

 
• Displaying glitch rates & other quality measures after common mode commences, not waiting until science mode - The state vector bit corresponding to "common mode" (feedback to laser) in S2 was defined to be on as soon as the servo loop was closed, but what was more relevant to DMT monitors was the completion of the CM feedback script, after all filter / power adjustments had been made. To avoid spurious triggers and misleading trends during the few minutes required for the script to complete, many DMT authors chose to report results only when in science mode. As a result, the operators did not have the glitch rate and band-limited RMS measures to look at when tweaking alignments in the period before declaring science mode. KR will talk to to DaveB and Chethan about getting another bit defined in the state vector for use in S3. John readily agreed to changing the critical DataQual monitor to look at CM data once the definition of CM is made more sensible.

 
• Adding more channels to glitchmon list -  Erik suggested that more channels be added to the glitchmon list, channels not presently in the standard reduced data set. Masahiro and John believe that available CPU time on the DMT machines at the site allow for many more channels to be measured, but that glitchmon would probably have to be split among several different monitors to avoid buffering troubles. Erik promised to send Masahiro a list of suggested optical lever channels, to get started. John noted that Rana has come up with a suggested reconfiguration of glitchmon channels. KR will make sure Masahiro receives a copy.

 
• Checking for hardware saturations in real time - Erik also suggested making checks for photodiode/ADC saturations in real-time (inferred saturations from reverse filtering of ASQ) of the sort that John is making now in re-analysis of the S2 data. John confirmed that he plans to make his present offline monitor into an online one. John also mentioned that there is a new feature being commissioned at LHO where the crate processor itself detects ADC saturations and sets a flag. Once it's working at LHO, it will be duplicated at LLO.

DMT Software Status Review:

 
• Status of DMT infrastructure

 
• John Zweizig: Version updates, etc.

A new DMT version (2.6) was installed recently at LHO, and nearly everything works. It uses gcc compiler 3.3,  has more autoconfigure/automake capability, and uses LDAS's version 0.7.0 FrameCPP. It also uses a new root version which has a bug that prevents ordinary cin/cout i/o in interactive mode. The CERN root team is working on the problem.
• Status of DMT monitors

 
• Dave Chin / Tim Bodiya: LockLoss and ServoMon/SpecMon (KR reporting)

No news on LockLoss & ServoMon, but SpecMon is a new monitor under development, meant eventually to supersede ServoMon in detecting spectral distortions, but with a much user-friendlier interface, including calibration options. Undergraduate Tim Bodiya has begun testing on the sand and fortress machines at LHO.
 
• Masahiro Ito: glitchMon

A new linear predictive filter developed by Shourov Chatterji is being implemented. The new filtering has the advantage of better whitening and line removal of the data prior to glitch hunting.
 
• Sergey Klimenko: LineMonitor and WaveMon

No update
 
• Szabi Marka: IRIG-B, TimeMon, and ShapeMon

No update
 
• Tim Olson: SpectrumArchiver

The spectral estimates computed by SpectrumArchiver are now better matched to those commonly used in the DTT, with 10 averages, using a Hamming window with 50% overlap. The stored resolution is 0.1 Hz binning for most channels. but for fast channels, the storage will be 0.1 Hz binning up to 1 kHz and 1.0 Hz binning above 1 kHz. The monitor presently stores spectra for all RDS channels, but Tim wishes to add many more channels, ideally all fast and all PEM channels. He will put together an estimate of the required data volume. For a single fast channel, one needs about 100 kB / hour, or 800 MB / year. The numbers stored will be amplitude spectral density. He has started working on an improved GUI interface in the DTT for retrieving the spectra.
 
• Adrian Ottewill: MTLineMon and CorrMon

No update
 
• Steve Penn / Vijay Chickarmane: BicoMon

No update
 
• Rauha Rahkola: eqMon

No current development
 
• Brian Stubbs: SuspensionMon

Brian is working on the data logger and an interface to the alarm class. KR will provide suggestions on formatting the html summary page.
 
• Patrick Sutton / Mike Ashley: SenseMonitor and RayleighMonitor

Aside from the SenseMon changed described above, Patrick is trying to track down a reported aliasing problem in RayleighMon and an apparent new instability when the monitor is run in its slow mode.
 
• Natalia Zotov: PTMon (report by e-mail)

Natalia is working with Vijay on tracking propagation of PEM glitches through IFO channels.
 
• John Zweizig: BitTest,  DataQual, HistCompr, PSLMon, and SegGener

John plans a number of upgrades for many of the monitors, including those discussed above.

A.O.B.

• Next detector characterization teleconference:  Thursday July 3, 2003 at 2:30 EDT

(avoiding the Friday holiday and the LSUG telecon at 1:30 EDT on Thursday)