I built the AmbitionBox salary scraper to collect that context into a consistent dataset. The Python pipeline discovers companies, extracts role-level salary records, writes batches and merges them into an analysis-ready CSV.

The parsing work

The most interesting part was the page’s embedded Nuxt data. It is not a plain JSON object: the payload uses a self-invoking JavaScript function, arguments and variable references to construct records.

The parser identifies the function boundaries, tracks nested delimiters and quoted strings, maps parameter names to argument values, and resolves the property assignments used by the job-profile records. It extracts the supported structure rather than executing the page’s JavaScript.

That distinction also defines a limitation: this is a parser for an observed payload format, not a general JavaScript interpreter. A change in the source representation can require an update.

A fallback with honest missing values

If the structured extraction returns no role records, the scraper falls back to the HTML table. That path can recover visible salary ranges and experience information, while leaving unavailable average-salary values empty.

An absent average should not become zero or an invented midpoint. Keeping it missing lets the later analysis distinguish between “this value was reported” and “this field was unavailable.”

Give the merged dataset a clear identity

The merge step uses company slug, role slug and location as the deduplication key. It keeps the last matching record in the concatenated input, converts numeric fields and sorts the output for inspection.

That is a concrete policy, not proof that a retained row is the newest observation. Adding an explicit collection timestamp would make that distinction easier to handle in future versions.

The workflow can collect available batch artifacts even when some jobs fail. A merged file therefore needs a completeness check before being treated as full coverage. I would add an expected-versus-received batch manifest and parser success counts as the next operational improvements.

What is complete

The repository contains the collection, parsing and merge pipeline. Its exploratory notebook and interactive dashboard are described as work in progress, so I do not present them as finished deliverables.

This is an educational data-engineering project. The dataset reflects the source’s salary reports; it is not verified payroll data or a representative survey of every employer. The portfolio value is the engineering: handling a nontrivial source format, preserving missing values and producing records that can be inspected and analysed.

Implementation details: payload parser, salary collection, merge logic, and batch workflow.

Crop Cure connects a grape-leaf classifier to WhatsApp. A user chooses a language, sends a photograph, and receives a response based on the predicted class. The project supports English, Hindi and Marathi response flows.

The workflow

My work here spans the classifier integration and the surrounding application: getting an image from a message into the model, interpreting the output, and turning that output into a usable reply.

The model has four labels: black rot, esca, leaf blight and healthy. That is a deliberately narrow scope. It is not a general classifier for every crop or every possible leaf condition.

A threshold needs an honest description

The implementation uses a 0.98 confidence threshold. Predictions below it return Unclassified.

That number is not 98% accuracy. It is a threshold applied to a model output. A high softmax score can still be wrong, especially when the photograph differs from the data used to train the model. Lighting, background, camera distance and an unsupported plant all make the input harder to interpret.

The fallback gives the application a way to avoid returning a class for every image. Whether it rejects enough unsuitable images requires evaluation with representative photographs; the public repository does not establish field accuracy.

Where the prototype stops

The repository describes the project as under development and not actively maintained. It is best understood as a research prototype, rather than a deployed agricultural diagnostic service.

Language preferences currently live in an in-memory dictionary, so they do not survive a process restart. Inference also runs synchronously inside the message-handling path. If I continued the project, I would persist conversation state, move inference into a controlled worker path, and evaluate class-specific errors and rejection behaviour on a held-out field dataset.

The interesting engineering lesson is how much of the product sits around the model. The preprocessing contract, supported inputs, fallback behaviour and message flow all determine whether the prediction can be used sensibly.

Implementation details: model and preprocessing and WhatsApp application.

FreightSense is my prototype for that workflow. It puts a rules-based calculation beside an LLM assessment, records both, and leaves the final decision with the operator.

What I built

The application combines a Python/FastAPI API, a small browser dashboard, a deterministic evaluation layer, Groq-hosted LLM calls and SQLite records. The repository also includes Docker and Cloud Run deployment configuration.

Keep the arithmetic outside the prompt

The deterministic layer calculates delay and exposure before the model sees the request. Its risk score combines delay severity, historical context, financial exposure and margin. The recommendation itself follows separate business rules: no delay can mean NO_ACTION; a feasible and economically acceptable intervention can mean EXPEDITE; otherwise the result can be DISCOUNT or MONITOR.

That distinction matters. A risk score describes the situation. It does not, by itself, prove that paying to expedite is sensible. The code checks conditions such as the shipping mode and the estimated intervention cost.

The LLM receives these calculations as context. The application parses its response, checks the recommendation label and bounds the confidence value. It also handles several failure paths, including API errors and invalid JSON, by returning the deterministic result with the LLM marked unavailable.

Disagreement is useful information

A model disagreeing with the rules should be visible. FreightSense stores the two assessments and flags the mismatch instead of quietly replacing one with the other.

The override endpoint lets a person record a different decision, a reason and outcome notes. Multiple overrides are retained as separate entries. That history makes it possible to revisit why an operator intervened, rather than seeing only the latest choice.

What the prototype establishes

The repository demonstrates the complete evaluation-and-review path. It does not establish a measured reduction in shipping costs. The financial figures are estimates, and the model’s confidence is a self-reported value, not a calibrated probability of correctness.

For a production version, my next priorities would be durable shared storage, access control, stronger validation of model outputs, and a labelled evaluation set that includes expensive mistakes and ambiguous cases. Those are more useful next steps than adding another model to the comparison.

Implementation details: deterministic rules, LLM evaluation, and API routes.

One small API

1
2
3
4
5

from loadfile import load

df = load("data/local.csv")
df = load("gs://my-bucket/data.parquet")
df = load("archive.zip", filename="sales.csv")

The function is named load(). The package also exports load_data() as a backwards-compatible alias.

fsspec selects the storage backend from the path prefix. The package selects the reader from the file extension, or an explicit format= argument, and passes reader options through to pandas. Cloud backends require their corresponding optional dependencies and credentials.

1
2

df = load("export.tsv", format="csv", sep="\t")
df = load("large.csv", fast=True, usecols=["id", "value"])

CSV, Parquet, JSON, Excel and Feather share the same interface. fast=True opts into Arrow-backed reading; it changes the reading defaults rather than promising a fixed speedup for every file.

ZIP files without another helper

A ZIP containing one supported data file returns a DataFrame. Multiple supported members return a dictionary keyed by filename. You can select one member by name or pass a list to load a subset.

The implementation reads ZIP contents into memory, so archive size still matters. The aim is a convenient reusable loader, not an out-of-core processing engine.

Source: public API and loading implementation.

This dashboard brings those questions into one Power BI report. It uses a historical consumer-complaints dataset containing more than 75,000 records, with views across products, geography and time.

Start with the questions

I built the dashboard to make these measures explorable together. Someone looking at one product can inspect its volume and response outcomes without rebuilding the analysis from a spreadsheet each time.

The repository includes the Power BI file, the source CSV and a dashboard image. That makes the analytical artifact available to inspect, rather than leaving the project at a screenshot alone.

Counts need context

A product with the most complaints is not automatically the worst-performing product. It may also have the most customers. The complaint dataset alone does not provide every denominator needed to make that comparison.

Rates need equally careful reading. A timely-response percentage and a dispute percentage answer different questions; neither is a substitute for understanding the underlying cases. Filters change the population being examined, so the selected product and period are part of the interpretation.

This is why I would present the report as a way to locate patterns worth investigating. It does not establish the cause of a complaint or prove that an operational intervention worked.

What I would add next

The next useful addition would be a short metric dictionary beside the report: each measure’s numerator, denominator, exclusions and treatment of missing values. I would also make the dataset’s coverage period and refresh status more prominent.

Those additions would help another person reproduce the reading of a chart. For an analytical portfolio project, that is a stronger improvement than adding more visuals to the same page.

The published result is an interactive report on historical data. No real-time refresh or measured business impact is claimed here.