hf jobs run mounts Hub repositories and Storage Buckets
Introduced 2026-03-25 · score 0.97
HOW TO USE IT
When a job needs to query a Hub dataset from mounted files, mount the dataset at `/data` and run DuckDB against `/data/**/*.parquet`.
from huggingface_hub import run_job, Volume
job = run_job(
image="duckdb/duckdb",
command=["duckdb", "-c", "SELECT * FROM '/data/**/*.parquet' LIMIT 5"],
volumes=[Volume(type="dataset", source="HuggingFaceFW/fineweb", mount_path="/data")],
)hf jobs run -v hf://datasets/HuggingFaceFW/fineweb:/data duckdb/duckdb duckdb -c "SELECT * FROM '/data/**/*.parquet' LIMIT 5"WHY THIS HELPS
Run a job that queries a Hub dataset directly from its mounted files.
REQUIREMENTS AND LIMITATIONS
- Version: v1.8.0
- run_job.volumes: List of `Volume` objects mounted in the job container.
- Volume.type (required): Volume type; `dataset` identifies a dataset repository.
- Volume.source (required): Source identifier, such as `HuggingFaceFW/fineweb`.
- Volume.mount_path (required): Absolute path inside the container; must start with `/`.
- -v, --volume: Mount a volume; Hub source format is `hf://[TYPE/]SOURCE:/MOUNT_PATH[:ro]`. Can be repeated. Dataset repos are mounted read-only.
- Prerequisite: Volume mounting requires `huggingface_hub` >= 1.8.0.
- Mounted volumes can access Hugging Face model, dataset, and Space repositories, as well as Storage Buckets.
FIRST-PARTY EVIDENCE
- https://github.com/huggingface/huggingface_hub/releases/tag/v1.8.0
- https://huggingface.co/docs/huggingface_hub/guides/jobs
- https://huggingface.co/docs/huggingface_hub/en/guides/cli
- https://huggingface.co/docs/huggingface_hub/main/package_reference/jobs
- https://huggingface.co/docs/hub/en/jobs-configuration
Check the installed version and the source before using it. Entry JSON