Engineering case study / AI integration
Local intelligence.
Turning public websites into structured company data.
01 / The problem
What needed solving.
Company websites contain useful industry and market-positioning information, but that information arrives as unstructured pages. The procurement product needed structured metadata that could live alongside its existing company records.
02 / How it fits together
- 01Public websites
- 02Secure scraper
- 03Ollama + Qwen
- 04PostgreSQL
03 / Engineering decisions
Run the model locally
I stood up Ollama and Qwen on a GPU environment. A local inference setup let the extraction pipeline run against a controlled model environment and connect to the rest of the data workflow.
Separate collection from extraction
The scraper collected public company pages, while prompt templates guided Qwen's extraction of industry and market-positioning metadata. Keeping the stages separate made the pipeline easier to inspect and maintain.
Finish at the database
The output was synced into PostgreSQL using Prisma, making the extracted metadata useful inside the product rather than leaving it in a separate experiment.
04 / Where it stands
Built the pipeline across approximately 110 public company websites during my Nest2Move internship. This describes the implemented scope; no extraction-accuracy or performance benchmark is claimed.