Skip to content

mecfs_bio.figures.key_scripts.pull_figures

Make the local figure directory match the manifest committed to git.

For each entry path -> sha256 in the manifest, the corresponding blob is fetched from the GitHub release (asset name = sha256) only if the local file is missing or has a different hash. Files under the figure directory that are not listed in the manifest are left alone unless prune=True is passed.

Functions:

  • pull_figures –

    Sync the local figure directory with the manifest by downloading any

Attributes:

DEFAULT_CHUNK_SIZE module-attribute 

DEFAULT_CHUNK_SIZE = 20

DEFAULT_CHUNK_WORKERS module-attribute 

DEFAULT_CHUNK_WORKERS = 4

logger module-attribute 

logger = structlog.get_logger()

pull_figures 

pull_figures(
    tag: str = FIGURE_GITHUB_RELEASE_TAG,
    repo_name: str = GH_REPO_NAME,
    fig_dir: Path = FIGURE_DIRECTORY,
    manifest_path: Path = FIGURE_MANIFEST_PATH,
    use_gh_cli: bool = True,
    prune: bool = False,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
    chunk_workers: int = DEFAULT_CHUNK_WORKERS,
)

Sync the local figure directory with the manifest by downloading any missing or out-of-date blobs from the GitHub release.

If prune is True, files under fig_dir that are not listed in the manifest are deleted.

Source code in mecfs_bio/figures/key_scripts/pull_figures.py 
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
def pull_figures(
    tag: str = FIGURE_GITHUB_RELEASE_TAG,
    repo_name: str = GH_REPO_NAME,
    fig_dir: Path = FIGURE_DIRECTORY,
    manifest_path: Path = FIGURE_MANIFEST_PATH,
    use_gh_cli: bool = True,
    prune: bool = False,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
    chunk_workers: int = DEFAULT_CHUNK_WORKERS,
):
    """
    Sync the local figure directory with the manifest by downloading any
    missing or out-of-date blobs from the GitHub release.

    If ``prune`` is True, files under ``fig_dir`` that are not listed in the
    manifest are deleted.
    """
    fig_dir.mkdir(parents=True, exist_ok=True)
    manifest = FigureManifest.load(manifest_path)

    if not manifest.figures:
        logger.debug(f"Manifest {manifest_path} is empty; nothing to download.")

    to_download: list[tuple[PurePath, str]] = []
    for rel_path, expected_hash in manifest.figures.items():
        dest = fig_dir / rel_path
        if dest.is_file() and sha256_of_file(dest) == expected_hash:
            logger.debug(f"{rel_path} is up to date; skipping download.")
            continue
        to_download.append((rel_path, expected_hash))

    if to_download:
        _download_blobs(
            to_download=to_download,
            tag=tag,
            repo_name=repo_name,
            fig_dir=fig_dir,
            use_gh_cli=use_gh_cli,
            chunk_size=chunk_size,
            chunk_workers=chunk_workers,
        )

    if prune:
        _prune_unmanifested(fig_dir=fig_dir, manifest=manifest)