Minutes of Detector Characterization Teleconference (June 18, 1999)

Present:

Caltech: Barish, Gustafson, Lazzarini, Majid, Yamamoto, Zweizig
Dublin: Ottewill
Hanford: Adhikara, Bhawal, Raab, Schofield, Sigg
Livingston: Giaime
Louisiana State: Johnson
Michigan: Riles
MIT: Daw, Penn, Shoemaker
Oregon: Brau
Penn State: Finn

Data Monitor Tool (John Zweizig):

John's outline (includes details not discussed during meeting, along with web links)
John has been concentrating lately on building Data Monitor Tool (DMT) root classes and templates, using 40-meter data for testing. See the above link for an example of a spectral strip chart.
He plans to make the root classes publicly available soon.
The background controlling process software was released in April.
It was decided during the meeting that the officially supported compiler for DMT will be the GNU C++ compiler, egcs (version 1.1.2 or higher). This makes the code usable on a large number of platforms while maintaining compatibility with the Framecpp library (LIGO's c++ version of the frame library).
The code can be run on any recently produced frame file. Albert Lazzarini mentioned that 1994 40-Meter data can be accessed via a CACR web server for DMT testing. A link to that server will be placed on the detector characterization bulletin board web page.
It was noted that DMT will NOT run on older-format frame files
Algorithm writers for DMT are urged to follow the LIGO software specification and style guide (a link to which will also be placed on the bulletin board) and to use the algorithm library routines as they become available (see next talk).

Algorithm Libray (Walid Majid):

Walid, Bruce and Sam are putting together the core of an algorithm library that conforms to the software specification guide. Although progress has been made on some of the primitive routines for data creation, destruction, copying, etc., important issues concerning supported data types and interfaces remain unresolved. An effort will be made in the nextweek to resolve these issues and begin coding in earnest.
These disagreements are driven in part by the decision earlier this year to write algorithms in procedural c code, which doesn't allow for the function overloading available in c++. The decision to use procedural c instead of object-oriented c++ was made in order to widen the pool of scientists able to write LIGO code.
Those LSC members who wish to get started immediately on DMT algorithms do not have to wait for this library to be completed or even its first version to be released, but they are urged to keep its future existence in mind when programming. Ultimately, we want all DMT code to conform to the LIGO standard that will prevail in LDAS software. For example, if using a private FFT or windowing algorithm, modularize it for later substitution of a standard version.
Walid et al are focussing on three high-level tasks: 1) power spectrum estimation; 2) Heterodyning; and 3) Decimation/smoothing to determine which primitive routines are needed most urgently.

Performance Characterization (KR):

Quite a number of volunteers have expressed interest in various aspects of performance characterization which is defined (roughly) to be characterization of stationary or quasi-stationary behavior. These expressions of interest are indicated in the detailed tables put together for the analysis white paper (see later item on agenda).
A smaller number have committed to delivering code that runs in the DMT on the timescale of this summer or fall:

Bruce Allen (UWM) & Adrian Ottewill(UWM/Dublin) - Tracking of lines and principal value decomposition to obtain channel-to-channel correlations. Bruce & Adrian visited Hanford in May and succeeded in installing GRASP and running it on the data stream. They plan to complete the DMT work this summer and will make another visit to Hanford soon. Adrian mentioned that GRASP has been recently revised to handle floating point numbers in the data, required in order to look at trend information.
Sam Finn & Soma Mukherjee (PSU) - Tracking of lines. They plan completion in late summer or early fall.
Warren Johnson & Bob Svoboda (LSU) - Tracking of lines. They plan completion during the summer.
Rana Adhikari (MIT) - Student working with Daniel Sigg at Hanford on time-frequency methods & plotting. Work will be carried out in the summer.
KR (Michigan) and Steve Penn (Syracuse) - "Gap filling". We will try to cover high-priority areas of performance characterization receiving little attention. Depending on who comes forward soon, we will try to cover some subset of estimation of seismic noise, stack vibrations, operational state description and non-Gaussian noise. Helping out on transient analysis (see below) may be appropriate.

Albert Lazzarini mentioned that a SURF student, Denis Petrovich, will be working with him this summer on poly-spectra studies of 40-Meter data

Transient Analysis (Fred Raab):

Again, quite a number of volunteers have expressed interest in this area, but none so far has committed to delivering code.
Fred hopes that with non-trivial data now becoming available at Hanford, more scientists will take an interest in writing & testing algorithms.
He will ask persons who have expressed interest in transient analysis to begin writing periodic reports (e.g., once/month) describing their activity.
He expressed concern that uncertainties in LIGO-I membership and access to LIGO-I data might be hindering more participation. Barry Barish stated that a clarified policy on LSC MOU's is in the works and should be available in the next few weeks.
Fred urged scientists to resurrect old analysis tools and adapt them to the DMT environment.

(Summary of discussion on data access deferred to Data Tape Technology agenda item below.)

Data Set Reduction (Jim Brau):

A small group at Oregon (JB, David Strom & Robert Schofield) are examining issues relevant to data set reduction. These include data tape technology, internet access to data at the sites, and compression techniques, including intelligent distillation. (Summary of discussion on data access deferred to Data Tape Technology agenda item below.)
This group is concerned not only with data reduction relevant to LDAS analysis, but also with customized data set reduction for scientists needing snippets of specific channel information for finite time duration. Some of the infrastructure is already available via the java data server written by Alex Ivanov of Caltech, and a version that can be used to write to disk files will be available in the next two weeks.
Compression factors of about 100 are thought to be realistic with minimal information loss. Albert pointed out that gzip has been shown to give lossless compression of about three.
Jim is looking for volunteers to write pieces of the data reduction infrastructure which the group is now defining.

Data Set Simulation (Sam Finn):

Again, there is much expressed interest but few volunteers. Only Sam and Ed Daw have committed to deliverable software.
Sam has written a number of noise generators that allow direct conversion to time series in matlab form, but hasn't yet written the code necessary for frame generation. His code allows mimicking of the expected LIGO noise spectrum.
Hiro Yamamoto pointed out that the End-To-End model (E2E) could readily incorporate Sam's algorithms and produce data in the required format. He and Sam will work on this integration.
The E2E code can be downloaded to home institutes, either as executable code for Solaris machines or as compilable code for those who have the latest GNU EGCs compiler (same needed for DMT code above). Interested persons are urged to try downloading and trying the E2E code, with comments to be sent to Hiro.
It was noted that Sam's code is a macro-model (i.e., based on parametrization of overall IFO behavior), useful for fast simulation and algorithm testing. E2E is a bottoms-up micro-model. useful for detailed comparisons between predicted and measured behavior.

Data Tape Technology (Daniel Sigg):

(This discussion actually took place in several parts during earlier agenda items)
In principle, data from the observatories is available over the internet on a T1 line now, but the limited bandwidth makes this useful for only relatively small data transfers.
Large data transfers from the observatories directly to most LSC institutes will require writing to tape for mailing.
Albert suggested that helping operate tape drives at the sites is a natural service task for LSC scientists on site to assist with, perhaps as part of shift taking.
A Sun-450 workstation ("size of small refrigerator") with four 400 MHz processors, 1 GB RAM and ample disk space has been ordered and will support an 8-mm tape drive (exabyte).
Jim Brau stated that the SONY AIT-2 technology might be most appropriate for home institutes, since a single tape can hold 50 GB. For reference, a day's full data from all three IFO's is of order one TB. A tape drive costs a few thousand dollars.
This is, in fact, a collaboration-wide issue that affects the other analysis groups too. Clearly a single agreed-upon and reasonably inexpensive technology is desirable.
To help compare the various options, Stuart Anderson has assembled a web page summarizing the relevant parameters (e.g., cost, capacity, xfer rates). David Strom and Ben Evans are also investigating tape technology options.
Interested persons are urged to look at Stuart's page and plan discussing this issue at the Stanford LSC meeting.

Analysis White Paper Contribution (KR):

A White Paper on LIGO data analysis is being prepared by the LSC for initial release this fall.
The White Paper is meant to give guidance to NSF on long-range funding needs and to give guidance to reviewers of NSF proposals.
It will contain tables of tasks / priorities / groups.
A small group (Rai Weiss + analysis groups chairs / liaisons) is preparing a draft for release before the Stanford meeting.
A first draft of the text and tables to be contributed by the detector characterization group has been released for consideration by this group.
Group members are urged to

Provide comments on the text and tables to Keith & Daniel by e-mail (or phone) before the Stanford meeting. We hope to have a nearly final version approved at that meeting. (Jim Brau pointed out already that the data set reduction subtable needs amplification; he will provide a revised version.)
Look carefully at the detailed task tables and please volunteer for tasks uncovered or covered by few scientists. Some tasks, such as tracking lines in the data, are amply covered. Current volunteers for such tasks may want to consider working elsewhere. It would be desirable to spread the effort more evenly.

LDAS at Hanford (Albert Lazzarini):

Albert quickly reviewed recent LDAS work at Hanford by Kent Blackburn, Stuart Anderson, Ed Maros and himself.
An ATM switch is now connected to the CDS frame builder, providing 612 Mb/s throughput.
The Sun 450 workstation has been benchmarked to give sustained 50 Mb/s throughput to the main data output trunk while using only 4% of the CPU.
Fiber optics have been installed to support the general computing switch. A T1 line provides internet access to data.
Data servers and general-purpose workstations and Linux boxes have been installed in the computer users room, next to the control room.
Private-network machines have also been installed for internal LDAS communication.

A.O.B.

Be sure to periodically check the working group bulletin board for important updates and miscellaneous links to useful documentation