Skip to content

mecfs_bio.build_system.wf.synapse_downloader

Synapse download capability

Classes:

SharedClientSynapseDownloader 

SharedClientSynapseDownloader()

Bases: SynapseDownloader

Downloads via a lazily authenticated Synapse client reused across calls.

Methods:

Source code in mecfs_bio/build_system/wf/synapse_downloader.py 
34
35
36
def __init__(self) -> None:
    self._client: synapseclient.Synapse | None = None
    self._login_lock = threading.Lock()

download 

download(
    synid: str, dest_dir: Path, expected_name: str
) -> Path
Source code in mecfs_bio/build_system/wf/synapse_downloader.py 
44
45
46
47
48
49
50
51
def download(self, synid: str, dest_dir: Path, expected_name: str) -> Path:

    files = synapseutils.syncFromSynapse(
        self._client_or_login(), synid, path=str(dest_dir)
    )
    assert len(files) == 1, f"expected one file for {synid}, got {len(files)}"
    assert files[0].name == expected_name, files[0].name
    return Path(files[0].path)

fetch_file_head 

fetch_file_head(synid: str, n_bytes: int) -> bytes
Source code in mecfs_bio/build_system/wf/synapse_downloader.py 
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
def fetch_file_head(self, synid: str, n_bytes: int) -> bytes:
    # Synapse entities are S3-backed; the file handle yields a pre-signed S3 URL that
    # honours HTTP Range, so we can pull just the first n_bytes instead of the whole
    # (often several-hundred-MB) file.
    client = self._client_or_login()
    entity = client.get(synid, downloadFile=False)
    file_handle_id = entity.dataFileHandleId
    download_info = client._getFileHandleDownload(file_handle_id, synid)
    presigned_url = download_info["preSignedURL"]
    response = requests.get(
        presigned_url,
        headers={"Range": f"bytes=0-{n_bytes - 1}"},
        timeout=60,
    )
    response.raise_for_status()
    return response.content

SynapseDownloader

Bases: ABC

Downloads a single file from synapse.org into a local directory.

Methods:

  • download –

    Download the single file at synid into dest_dir, assert its name equals

  • fetch_file_head –

    Return the first n_bytes of the file at synid WITHOUT downloading the whole

download abstractmethod 

download(
    synid: str, dest_dir: Path, expected_name: str
) -> Path

Download the single file at synid into dest_dir, assert its name equals expected_name, and return its local path.

Source code in mecfs_bio/build_system/wf/synapse_downloader.py 
19
20
21
22
@abstractmethod
def download(self, synid: str, dest_dir: Path, expected_name: str) -> Path:
    """Download the single file at synid into dest_dir, assert its name equals
    expected_name, and return its local path."""

fetch_file_head abstractmethod 

fetch_file_head(synid: str, n_bytes: int) -> bytes

Return the first n_bytes of the file at synid WITHOUT downloading the whole file. Used to read a tiny prefix (e.g. a tar header + the first rows of the first gzip member) from a large entity.

Source code in mecfs_bio/build_system/wf/synapse_downloader.py 
24
25
26
27
28
@abstractmethod
def fetch_file_head(self, synid: str, n_bytes: int) -> bytes:
    """Return the first n_bytes of the file at synid WITHOUT downloading the whole
    file. Used to read a tiny prefix (e.g. a tar header + the first rows of the first
    gzip member) from a large entity."""