Setting Up Jupyter

IGWN JupyterHub

# 1. Clone xpypeline
git clone git@git.ligo.org:gwdc/xpypeline.git        # SSH
git clone https://git.ligo.org/gwdc/xpypeline.git   # HTTPS
# On LDAS, a local clone is available:
git clone /home/angus.gray-weale/src/cas/ligo/xpypeline
cd xpypeline

# 2. Create the conda environment (use -p to choose the location)
eval "$(mamba shell hook --shell bash)"   # if mamba activate doesn't work
mamba env create -f environment.yml -p ~/opt/xp
mamba activate ~/opt/xp

# 3. Build and install
CMAKE_ARGS="-DCMAKE_DISABLE_FIND_PACKAGE_PkgConfig=ON" \
  pip install -e . --no-build-isolation

Register the Jupyter Kernel

# 4. Verify the installation
python -c "from xpypeline import isodd; print('xpypeline imports OK')"
python -c "from xpypeline.fastcluster import fastlabel; print('C++ extensions OK')"

# 5. Register the Jupyter kernel
python -m ipykernel install --user --name xpypeline --display-name "X-Pypeline"

• The X-Pypeline kernel will appear in IGWN JupyterHub
• Create a new notebook and select it from the kernel menu

Installation

Using the conda environment

# Clone the repository
git clone git@git.ligo.org:gwdc/xpypeline.git        # SSH
git clone https://git.ligo.org/gwdc/xpypeline.git   # HTTPS
# On LDAS, a local clone is available:
git clone /home/angus.gray-weale/src/cas/ligo/xpypeline
# On OzStar:
git clone /fred/oz449/pipeline/xpypeline
cd xpypeline

# Create the environment (use -p to choose the location)
eval "$(mamba shell hook --shell bash)"   # if mamba activate doesn't work
mamba env create -f environment.yml -p ~/opt/xp
mamba activate ~/opt/xp

The environment.yml provides everything: Python 3.11, conda C/C++ compilers (GCC 15.x), cmake, numpy, scipy, gwpy, h5py, matplotlib, jupyterlab, and xpypeline itself.

Building the C++ Extensions

# On LIGO clusters: skip optional lalinspiral extension
CMAKE_ARGS="-DCMAKE_DISABLE_FIND_PACKAGE_PkgConfig=ON" \
  pip install -e . --no-build-isolation

# Verify
python -c "from xpypeline import isodd; print('xpypeline imports OK')"
python -c "from xpypeline.fastcluster import fastlabel; print('C++ extensions OK')"

• Builds 10 C++ extensions via nanobind
• The conda compilers install as x86_64-conda-linux-gnu-gcc — CMake finds them automatically
• The optional lalinspiral extension is not needed for core functionality

Detailed instructions in docs/ligo_build.md

Example: GRB Sensitivity

Estimating detector response to a GRB

Given a GRB trigger time, sky position, and detector network:

1. Construct a search sky grid covering the position uncertainty
2. Compute antenna response patterns for each detector
3. Estimate the SNR of a gravitational wave burst

Based on sample_grb_notebook.ipynb — Python port of sample_grb_script.m
Available in the notebook repo: git.ligo.org/gwdc/xpypeline-notebooks

Step 1: Define Inputs

import numpy as np
from xpypeline.tildedelimstr2numorcell import tildedelimstr2numorcell
from xpypeline.xmakeskygrid import xmakeskygrid
from xpypeline.radectoearth import radectoearth
from xpypeline.antennaPatterns import antennaPatterns
from xpypeline.gwbenergy import gwbenergy
from xpypeline.SRD import SRD

# GRB trigger time and sky localisation
gps         = 1454920000       # trigger time [sec]
ra_ctr_deg  = 231.7            # right ascension [deg]
dec_ctr_deg = -34.1            # declination [deg]
sigma_deg   = 5.0              # 1-sigma sky position uncertainty [deg]

# Detector network and noise spectra
network_str = 'H1~L1~V1'
spectra_str = 'aLIGO~aLIGO~aVirgo'

# Signal properties
r        = 1e8                 # distance [pc]
f0       = 100                 # frequency [Hz]
Egw      = 0.01               # energy emission [solar masses]
emission = 'isotropic'

Step 2: Build Sky Grid

# Convert tilde-delimited network string to list
network = tildedelimstr2numorcell(network_str)  # ['H1', 'L1', 'V1']
spectra = tildedelimstr2numorcell(spectra_str)  # ['aLIGO', 'aLIGO', 'aVirgo']

# Create sky grid covering the GRB uncertainty region
n_sigma   = 2        # grid covers 2-sigma region
delay_tol = 5e-4     # delay tolerance [sec]

ra_search, dec_search, prob, area, cov = xmakeskygrid(
    str(ra_ctr_deg), str(dec_ctr_deg), str(gps), str(sigma_deg),
    str(n_sigma), network_str, str(delay_tol))

print(f"Sky grid: {len(ra_search)} points covering {cov[0]:.1f} deg")
# Sky grid: 98 points covering 11.6 deg

Step 3: Antenna Response

# Convert to Earth-fixed coordinates
phi_ctr, theta_ctr = radectoearth(ra_ctr_deg, dec_ctr_deg, gps)
phi_grid, theta_grid = radectoearth(ra_search, dec_search, gps)

# Antenna responses at the GRB position
sky_ctr = np.array([[np.squeeze(theta_ctr), np.squeeze(phi_ctr)]])
Fp, Fc, _ = antennaPatterns(network, sky_ctr)
Frss = np.sqrt(Fp**2 + Fc**2).ravel()

# Antenna responses over the search grid
Fp_grid, Fc_grid, _ = antennaPatterns(
    network, np.column_stack([theta_grid, phi_grid]))
Frss_grid = np.sqrt(Fp_grid**2 + Fc_grid**2)

Step 4: Energy and SNR

# Determine hrss amplitude of the signal
hrss_nominal = 1.0
Egw_nominal, _ = gwbenergy(r, hrss_nominal, f0, emission)
hrss = hrss_nominal * (Egw / Egw_nominal)**0.5

# Detector noise spectra at signal frequency
S = np.array([float(SRD(s, f0)[0]) for s in spectra])

# SNR for each detector
SNR = Frss * hrss / np.sqrt(S)

           RSS antenna response       SNR
Detector   grid centre    worst     (centre)
--------   -----------    -----     --------
  H1       0.4418        0.2922   7.4389
  L1       0.3508        0.1949   5.9058
  V1       0.4003        0.2667   4.8333
             Coherent network SNR: 10.6572

Running the Full Pipeline

xdetection

Calling xdetection

from xpypeline.xdetection import xdetection

skyPositions, likelihoodMap, skyPositionIndex, \
sourcePositions, sourceLikelihoodMap, sourcePositionIndex, \
spectrogram = xdetection(
    parameterFileName='input/parameters_on_source_0.txt',
    jobNumberString='0',
    outputDirectory='output/',
    injectionNumberString='0'
)

Parameter File

channelFileName:input/channels.txt
frameCacheFile:input/framecache.txt
eventFileName:input/event_on_source.txt
skyPositionList:[0.772832,1.397544]
skyCoordinateSystem:earthfixed
likelihoodtype:circenergy,circinc,circnullenergy,circnullinc
analysistimes:0.5
blocktime:64
onsourcebeginoffset:-28
onsourceendoffset:28
minimumfrequency:64
maximumfrequency:500
samplefrequency:1024
whiteningtime:1
seed:1235
makesimulatednoise:aLIGO

Same format as MATLAB X-Pipeline — existing parameter files work unchanged

GRB Pipeline Stages

• Each stage reads and writes MAT files — same format as MATLAB
• Can mix Python and MATLAB stages (same file format)
• All stages ported and validated

Running a GRB Search

# Stage 1: Detection (typically many parallel jobs)
for job in $(seq 0 139); do
  python -m xpypeline.xdetection \
    input/parameters_on_source_0.txt $job output/
done

# Stage 2: Merge results
python -m xpypeline.xmerge output/

# Stage 3: Veto tests (21 veto combinations)
python -m xpypeline.xmakegrbwebpage output/

# Stage 4: Tune detection threshold
python -m xpypeline.xtunegrbwebpage output/

# Stage 5: Closed-box result
python -m xpypeline.xclosedbox output/

In practice, stage 1 runs as parallel HTCondor or Slurm jobs

Full Example Run

• A complete end-to-end GRB search is available at /fred/oz449/pipeline/
• The integration repo ligo:gwdc/xpypeline-integration.git contains the scripts used to run and validate the full pipeline
• All 6 stages, 140 detection jobs, 21 veto combinations
• Field-by-field comparison of every output file against MATLAB

# Clone the integration test suite
git clone https://git.ligo.org/gwdc/xpypeline-integration.git   # HTTPS
git clone git@git.ligo.org:gwdc/xpypeline-integration.git        # SSH

Tutorial Notebooks

git clone https://git.ligo.org/gwdc/xpypeline-notebooks.git   # HTTPS
git clone git@git.ligo.org:gwdc/xpypeline-notebooks.git        # SSH

(*) Requires MATLAB engine for Python

Running the Notebooks

# Clone and set up
git clone https://git.ligo.org/gwdc/xpypeline-notebooks.git   # HTTPS
git clone git@git.ligo.org:gwdc/xpypeline-notebooks.git        # SSH
cd xpypeline-notebooks

# Install xpypeline and register the Jupyter kernel
./setup.sh

# Verify all notebooks execute successfully
./verify.sh

• setup.sh clones xpypeline, creates the conda environment, and registers the kernel
• verify.sh runs each notebook with jupyter nbconvert --execute and reports pass/fail

chocripple: Pipeline Setup

The pre-xdetection pipeline — automates everything before the search runs.

• Reads GRB configuration files and sets up the analysis
• Segment and data quality queries (ligo.segments, gwpy)
• Frame data discovery
• Sky position grid generation
• Injection file creation
• Workflow generation via Pegasus WMS (supports both HTCondor and SLURM)

git clone https://git.ligo.org/gwdc/chocripple.git   # HTTPS
git clone git@git.ligo.org:gwdc/chocripple.git        # SSH

chocripple: Target Environments