I built loadfile to stop copying the same file-loading code between scripts. It provides one entry point for tabular data, whether the file is on local disk or in cloud storage.
One small API
from loadfile import load
df = load("data/local.csv")
df = load("gs://my-bucket/data.parquet")
df = load("archive.zip", filename="sales.csv")
The function is named load(). The package also exports load_data() as a backwards-compatible alias.
fsspec selects the storage backend from the path prefix. The package selects the reader from the file extension, or an explicit format= argument, and passes reader options through to pandas. Cloud backends require their corresponding optional dependencies and credentials.
df = load("export.tsv", format="csv", sep="\t")
df = load("large.csv", fast=True, usecols=["id", "value"])
CSV, Parquet, JSON, Excel and Feather share the same interface. fast=True opts into Arrow-backed reading; it changes the reading defaults rather than promising a fixed speedup for every file.
ZIP files without another helper
A ZIP containing one supported data file returns a DataFrame. Multiple supported members return a dictionary keyed by filename. You can select one member by name or pass a list to load a subset.
The implementation reads ZIP contents into memory, so archive size still matters. The aim is a convenient reusable loader, not an out-of-core processing engine.
Source: public API and loading implementation.