It's essential then to maintain close interaction between this working group and the LIGO Global Diagnostics Systems (GDS) Group, both to minimize duplication of effort and to allow efficient contribution of effort to data analysis from the non-LIGO-LAB LSC members.
An important role of this working group then is to guide LSC members toward useful avenues of pursuit and to work with the LIGO Lab to reduce the overhead required to contribute effectively. Although plans are still evolving, here is a summary of three stages of data analysis, followed by an overview of each stage. These overviews are relative crude descriptions of fairly involved processes. Following some of the links given below will lead one to more detailed (and accurate) information.
It is mainly in the second stage where the LSC detector characterization group is expected to contribute, but various members will likely be active in all stages.
Summary of three stages of data analysis:
Online diagnostics will be stripped-down, fast algorithms that both monitor the data passively (e.g., recording statistics and anomalous channel information into the LIGO Meta-database) and invasively (e.g., injecting swept sine waves at test points to measure transfer functions). By and large, algorithms running online look directly at reflective memory locations in the DAQ readout chain. (As a result, there is some natrual overlap and much coordination needed between the GDS and Control & Data Systems (CDS) groups. Because of the special constraints on the online code and the necessity for it to be bomb-proof, it will be difficult for LSC members to contribute directly to this software without spending substantial time on-site.
Offline Processsing, Monitoring & Triggering:
Offline (but on-site) analysis of data will occur mainly on Sun workstations and will include monitoring of data for transients & anomalies, generating additional records to be stored in the meta-database, creation of a standard Lightweight Data Format, and creation of additional, special-purpose reduced data sets.
The monitoring tasks and generation of reduced data sets are natural areas in which the LSC detector characterization group should contribute. Most of the basic tools will be written in C or C++ language , but there will also be Application Program Interface (API) packages allowing access to the data via such programs as ROOT and MATLAB and perhaps others. An important part of the offline analysis infrastructure will be the Data Monitoring Tool which will provide rapid access to data as it streams from the detector. Designed to continuously monitor the data, it will generate alarms to be sent to the control room and generate less urgent triggers to be logged into the meta-database.
Basic monitoring algorithms (C and C++) are in progress and should serve as useful templates for interested contributors of more advanced algorithms. The Data Monitor Tool will make use of (among others) the Frame Class Library (FCL) for extracting raw data from frames and the Algorithm Class Library (ACL) now under development. The latter will contain efficient, standardized numerical algorithms for data manipulation (e.g., FFT's, autocorrelations) written (and available as) C procedures, but embedded in a higher-level C++ environment.
The alternative access to data via Root and Matlab provides two rich and well developed frameworks for interactive data manipulation and graphics. In addition, Matlab already contains impressive signal processing algorithms, and the ACL algorithms will be designed to run under Root which includes a C++ interpreter and can be linked to other libraries.
It is not yet clear how best to incorporate the efforts of off-site
LSC algorithm developers into the above framework. For example, should
an experienced Matlab user stick with that environment for development
and worry later about incorporation into the Data Monitor Tool? Should
one jump directly into C or C++, using
the ACL alogrithms and perhaps Root for graphic displays? The second
approach is more likely to be of direct, immediate benefit to LIGO, but
it's too early to know for sure. Matlab , the GEO-created TRIANA
package, or perhaps some other package may yet prove to be the best route
to usable and used monitoring algorithms. For now, though, LSC members
are encouraged to go the C/C++/Root route.
Archiving & Astrophysical Searches:
The Center for Advanced Computing
Research (CACR) will serve as the archive of LIGO data from both sites.
The LIGO Data
Analysis System (LDAS) will provide access to the data via computing
engines accessible remotely over the internet. The bulk of astrophysical
data analysis (i.e., searches for sources of gravitational radiation)
will take place on these computers (or at outside institutes using highly
reduced data sets). Ideally, physicists analyzing the data will have
in hand metadata produced by on-site diagnostics and detector characterization
algorithms, allowing, for example, optimal search filters to be used at
once. More likely, though, residual detector effects missed on-site will
be found from time to time. A simple mechanism for migrating useful detector
algorithms developed at this level back upstream to the offline or even
online would be highly desirable. Consequently, an effort will be made
to use the same basic algorithm tools both offline and in CACR-base analysis.