Skip to content

mecfs_bio.build_system.task.protein_coloc.csf_window_fetcher

DenseWindowFetcher for the Western et al. 2024 CSF pQTL GWAS: reads trait-locus windows of one aptamer's GWAS Catalog harmonised summary statistics by tabix range requests, without downloading the file.

Every study has a bgzipped, tabix-indexed GRCh38 harmonised file with a header line and the columns in CSF_HARMONISED_COLUMNS. Contigs are "1".."22": there is no chrX, so a window on a chromosome the index does not list yields an empty frame. Missing values are written "NA" and become nulls. Effect allele and frequency are reported as the file has them; orientation to the reference genome happens downstream, as for every source.

The tabix index is downloaded into the call's scratch directory and handed to pysam explicitly; left to itself, htslib would save every remote index into the process's working directory. Failures to download the index (including a stalled connection, after timeout_s) or to open or read the source surface as OSError, so the coloc task's retry wrapper retries them; a region pysam rejects is a programming error and stays a ValueError, failing at once.

Classes:

Attributes:

CSF_HARMONISED_COLUMNS module-attribute 

CSF_HARMONISED_COLUMNS: Final = (
    "chromosome",
    "base_pair_location",
    "effect_allele",
    "other_allele",
    "beta",
    "standard_error",
    "effect_allele_frequency",
    "p_value",
    "variant_id",
    "rsid",
    "neg_log_10_p_value",
    "rs_id",
    "n",
    "hm_coordinate_conversion",
    "hm_code",
)

CsfWindowFetcher

aptamers: the manifest rows of every aptamer that may be requested, looked up by seq id (the coloc protein id). resolver: where an aptamer's harmonised file lives; the GWAS Catalog FTP by default. index_fetcher: brings the tabix index into the scratch directory.

Methods:

Attributes:

aptamers instance-attribute 

aptamers: tuple[CsfAptamerFile, ...]

index_fetcher class-attribute instance-attribute 

index_fetcher: IndexFetcher = UrlIndexFetcher()

resolver class-attribute instance-attribute 

resolver: HarmonisedLocationResolver = (
    GwasCatalogHarmonisedResolver()
)

fetch 

fetch(
    protein_id: str,
    windows: Sequence[LocusWindow],
    scratch_dir: Path,
    wf: WF,
) -> dict[str, pl.DataFrame]
Source code in mecfs_bio/build_system/task/protein_coloc/csf_window_fetcher.py 
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
def fetch(
    self,
    protein_id: str,
    windows: Sequence[LocusWindow],
    scratch_dir: Path,
    wf: WF,
) -> dict[str, pl.DataFrame]:
    aptamer = self._aptamer(protein_id)
    location = self.resolver(aptamer.accession)
    index_dir = scratch_dir / f"tabix_index_{protein_id}"
    index_dir.mkdir(parents=True, exist_ok=True)
    index = self.index_fetcher(location.index, index_dir)
    try:
        return _fetch_windows(location.data, index, windows)
    finally:
        shutil.rmtree(index_dir)

GwasCatalogHarmonisedResolver

HarmonisedLocation

URL or local path of a bgzipped harmonised file (data) and of its tabix index.

Attributes:

data instance-attribute 

data: str

index instance-attribute 

index: str

HarmonisedLocationResolver

Bases: Protocol

IndexFetcher

Bases: Protocol

UrlIndexFetcher

Downloads the index (a few MB) with urllib; URLError and socket timeouts are OSErrors. timeout_s: give up on a connection that sends nothing for this long.

Attributes:

timeout_s class-attribute instance-attribute 

timeout_s: float = 60.0