I built loadfile to stop copying the same file-loading code between scripts. It provides one entry point for tabular data, whether the file is on local disk or in cloud storage.

One small API

1
2
3
4
5

from loadfile import load

df = load("data/local.csv")
df = load("gs://my-bucket/data.parquet")
df = load("archive.zip", filename="sales.csv")

The function is named load(). The package also exports load_data() as a backwards-compatible alias.

fsspec selects the storage backend from the path prefix. The package selects the reader from the file extension, or an explicit format= argument, and passes reader options through to pandas. Cloud backends require their corresponding optional dependencies and credentials.

1
2

df = load("export.tsv", format="csv", sep="\t")
df = load("large.csv", fast=True, usecols=["id", "value"])

CSV, Parquet, JSON, Excel and Feather share the same interface. fast=True opts into Arrow-backed reading; it changes the reading defaults rather than promising a fixed speedup for every file.

ZIP files without another helper

A ZIP containing one supported data file returns a DataFrame. Multiple supported members return a dictionary keyed by filename. You can select one member by name or pass a list to load a subset.

The implementation reads ZIP contents into memory, so archive size still matters. The aim is a convenient reusable loader, not an out-of-core processing engine.

Source: public API and loading implementation.