MASc, Software Engineering · Ontario Tech University
AI engineer & researcher working on trustworthy NLP for Requirements Engineering — LLM governance, knowledge discovery, and empirical software engineering.
MASc graduate (Ontario Tech, 2026) working at the intersection of NLP and Requirements Engineering (RE). My thesis presents TRAC-RE — a Traceable, Reliable, Auditable, and Contextual framework for automated requirements engineering, structured as four interconnected phases that take raw stakeholder prose to structured, audit-grade knowledge artifacts: context-aware requirement identification, audit-grade keyword extraction, governed LLM parsing, and implicit keyword discovery. The framework is grounded in an industrial FinTech/SaaS corpus of 110 requirements and 1,997 High-Level JSON (HLJ) artifacts. The thesis, defended in May 2026, argues that governance and distributional alignment — not raw model scale — are the primary levers for trustworthy RE automation; its governed-parsing phase is published at CASCON 2025 (IEEE).
Thesis
MASc thesis · Defended (May 2026)
MASc, Software Engineering · Ontario Tech University
Supervisor: Prof. Sanaa Alwidian
TRAC-RE is a framework for automated requirements engineering (RE) built from four interconnected phases. Together they take raw stakeholder prose to implicit domain-knowledge discovery — spanning context-aware requirement identification, audit-grade keyword extraction, governed LLM parsing into structured JSON, and multi-signal implicit keyword discovery, all grounded in an industrial FinTech and SaaS corpus.
The four phases
Sentence-level classification with structured local-context features. F1 improved from 0.664 to 0.894 on 110 real-world FinTech and SaaS documents; 4K domain-aligned samples (F1=0.894) beat 15K mixed (F1=0.883) with 73% fewer samples.
Perfect-precision extraction that also surfaced a structural coverage ceiling of 1.5 canonical keywords per artifact, motivating the implicit-discovery phase.
Multi-model parsing into confidence-scored High-Level JSON with a versioned tag-governance stack. Tag precision improved from 0.657 to 0.897 (v2); +0.31 F1 over GPT-4.1.
A multi-signal engine — neighbor transfer, graph walks over an enriched 275,164-edge co-occurrence graph, and UMAP+HDBSCAN cluster-gap detection — grounded in a 13,725-entry domain dictionary, with full per-keyword provenance.
The throughline across all four phases: governance and distributional alignment, not raw model scale, are the primary levers for trustworthy RE automation. LLMs are powerful but don't belong in every role — in TRAC-RE they parse and they tiebreak, but they never generate ground-truth knowledge without a traceable, auditable trail.A bounded LLM-as-judge design: LLMs never generate keywords, only tiebreak the borderline band, preserving auditability.
What the thesis asks
| RQ | Question | Phase | Finding |
|---|---|---|---|
| RQ1 | Does pipeline governance improve tag reliability? | Phase 3 — Governed LLM Parsing | v2 reaches 0.95 precision vs 0.77 at v1; +0.31 F1 over GPT-4.1. [1]Improving Reliability of LLMs in RE with Structured Confidence & Tag GovernanceDvip Patel, Sanaa AlwidianCASCON 2025, 2025Open Research Object · Research Object Reviewed |
| RQ2 | Does local context improve requirement identification? | Phase 1 — Requirement Identification | +16 F1 points at k=1; context-awareness is necessary, not optional. |
| RQ3 | Does feature structure matter beyond raw context? | Phase 1 — Requirement Identification | Naive concat drops F1 to 0.509; cosine divergence is the critical signal. |
| RQ4 | Does domain alignment beat scale? | Phase 1 — Requirement Identification | 4K domain instances (F1=0.894) beat 15K mixed (F1=0.883). |
| RQ5 | What is the explicit-keyword coverage ceiling? | Phase 2 — Audit-Grade Keyword Extraction | 1.5 kw/HLJ after normalization; Jaccard < 0.11 across extractors. |
| RQ6 | Can implicit keywords be discovered with traceability? | Phase 4 — Implicit Keyword Discovery | All 3 signals produce candidates at scale with per-signal provenance. |
| RQ7 | What is each signal's contribution? | Phase 4 — Implicit Keyword Discovery | Signal A dominates volume; graph enrichment is essential for Signal B. |
Selected figures
Four results carry the argument. Each figure is drawn from numbers reported in the thesis and the CASCON paper — no rescaled axes, every bar starts at zero, and the result each figure is about is set in colour.
| Configuration | F1 |
|---|---|
| Baseline | |
| Naive context concatenation | |
| Final configuration |
| Configuration | Training samples | F1 |
|---|---|---|
| Domain-aligned | ||
| Mixed |
| Configuration | Tag precision |
|---|---|
| v0 | |
| v2 |
| Configuration | F1 (v2) |
|---|---|
| Opus 4 | |
| Meta-70B | |
| GPT-4.1 |
What is claimed — and what is not
Contribution 1: TRAC-RE: a four-phase framework that establishes empirical and architectural foundations for reliable, structured, and auditable automated requirements engineering.
Contribution 2: A modular multi-model LLM parsing phase producing confidence-scored High-Level JSON (HLJ) artifacts with versioned tag governance (v0 → v2 → v3 in progress); tag precision 0.657 → 0.897.Threat to validity. Circular ground truth in the parsing and extraction phases: v2 pipeline output acts as its own reference with a 5% SME-audited sample.
Contribution 3: A context-aware requirement identification phase showing structured local-context features contribute +16 F1 points — ~70% of all gains across configurations.Threat to validity. Corpus is FinTech/SaaS only; generalization to safety-critical, healthcare, and automotive domains is untested.
Contribution 4: A multi-signal implicit keyword discovery phase combining neighbor transfer, graph walks over an enriched 275,164-edge co-occurrence graph, and UMAP+HDBSCAN cluster-gap detection, all with full provenance.Threat to validity. The implicit-discovery phase evaluation is a research preview — ablations across hop depth, signal composition, context window, and dictionary impact are specified and in progress.
Contribution 5: A bounded LLM-as-judge design: LLMs never generate keywords, only tiebreak the borderline band, preserving auditability.Threat to validity. No gold standard for implicit keywords; LLM-as-judge with 5% human calibration (Cohen's κ) is the planned proxy.
Contribution 6: Evidence that distributional alignment beats scale: 4K domain-only instances (F1=0.894) outperform 15K mixed (F1=0.883) using 73% fewer samples.Threat to validity. All experiments used a single random seed (42); relative comparisons remain valid.
Future work
The Research, Interactive
Most research lives in a PDF. Mine runs. Below is a short, guided walkthrough of the actual pipeline behind the thesis and its CASCON paper — in 2 steps. You'll see how the pieces connect, tune the knobs that drive the results, and watch the core finding play out on real requirement text. No prior context needed — each step explains itself.
Step 1 of 2 · What am I looking at?
A live map of how my work fits together — each circle is the thesis, the published paper, a research question, a finding, a dataset, or a future direction, and the lines show how they connect. The thesis anchors RQ2–RQ7; the CASCON paper answers RQ1. Click (or Tab to) any node to read what it is. Drag to rearrange, scroll to zoom.
Step 2 of 2 · Now make it yours
These are the real knobs from my experiments — thresholds, models, training modes. Move a slider and the metrics (F1, precision, recall) update instantly from pre-computed results of actual runs. This is the ablation study, made playable.
Adjust parameters to explore pre-computed results
Results
F1
84.7%
Precision
94.9%
Recall
79.9%
Confusion Matrix
Want the full control room?
Everything above — the map, the knobs, the pipeline stages, and a simulated peer-reviewer that critiques the methodology — in one full-screen workspace. This is the closest thing to sitting next to me while I run experiments.
Selected work
Dvip Patel, Sanaa Alwidian · CASCON 2025, 2025
A modular multi-model LLM pipeline that converts raw stakeholder requirements into High-Level JSON (HLJ) artifacts with confidence-scored fields, paired with a versioned tag-governance system that catches prompt-leak exploitation, hallucinated tags, and low-agreement outputs before they reach downstream stages.
Where I've worked
Remote · Part-time
Oshawa, ON · Supervisor: Prof. Sanaa Alwidian
Remote · Part-time
Remote
Built & shipped
Gmail-native communication intelligence layer — one ingestion pipeline, six analysis modules.
Built at a 48-hour hackathon. Turns unstructured email into structured, actionable intelligence: a single ingestion + parsing core (Claude classifier, entity extractor, temporal indexer) feeds six independent modules — financial waste detection, relationship decay signals, contract/obligation tracking, follow-up commitments, a RAG-based institutional memory, and health-admin tracking. One parse, many consumers — no redundant API calls, and an entity graph that compounds over time. Currently being taken from hackathon build to a live product.
Autonomous 10-step AI migration agent for legacy Laravel codebases.
CLI-driven migration agent that ingests legacy Laravel code and runs analysis → planning → transformation → validation → self-healing loops. Processes ~10k LOC per run at under $80 cost, with 85–90% automated issue resolution.
A Traceable, Reliable, Auditable, and Contextual framework for automated Requirements Engineering — structured parsing, context-aware classification, and implicit keyword discovery.
The research framework behind my MSc thesis at Ontario Tech University. **TRAC-RE** — Traceable, Reliable, Auditable, and Contextual — converts raw stakeholder prose into audit-grade knowledge artifacts across four interconnected phases: 1. **Phase 1 — Context-aware requirement identification** — Sentence-level classification with structured local-context features. F1 improved 0.664 → 0.894 on 110 FinTech/SaaS documents; 4K domain-aligned samples (F1=0.894) beat 15K mixed (F1=0.883). 2. **Phase 2 — Audit-grade keyword extraction** — Perfect-precision extraction that surfaced a structural coverage ceiling of 1.5 canonical keywords per artifact, motivating the implicit-discovery phase. 3. **Phase 3 — Governed LLM parsing (HLJ)** — Multi-model parsing (GPT-4.1, Claude Opus 4, Meta-70B) into confidence-scored High-Level JSON with versioned tag governance (v0 → v2); tag precision 0.657 → 0.897. Supporting paper published at CASCON 2025. 4. **Phase 4 — Implicit keyword discovery** — A 5-stage engine over a 275,164-edge enriched co-occurrence graph, with UMAP+HDBSCAN cluster-gap detection and a bounded LLM-as-judge tiebreaker. **Cross-cutting infrastructure** — 1,997 HLJ artifacts, a 13,725-entry domain dictionary, a FAISS index over 768-dim SBERT vectors, full per-decision audit logging, and drift monitoring across pipeline runs. The framework's central argument: governance and distributional alignment, not raw model scale, are the primary levers for trustworthy RE automation.
For the full engineering breakdown — story, stack, and impact — see the corporate view →
The toolkit
Technical competencies
Requirements Engineering, NLP for Software Engineering, LLM Governance, Knowledge Discovery, Empirical SE
Python, TypeScript / JavaScript, PHP (Laravel), SQL, Bash
PyTorch, HuggingFace Transformers, sentence-transformers, SBERT / MPNet, MiniLM, DeBERTa-v3, KeyBERT, RAKE, FAISS, UMAP + HDBSCAN, LoRA Fine-tuning, RAG Pipelines, Anthropic / Claude API, MCP, Aider
React, Next.js, Tailwind CSS, Zustand
Laravel, Flask, REST APIs, Microservices, RBAC, Docker, AWS, CI/CD, GitHub Actions
Vitest, fast-check (property-based), Testing Library
Education
Thesis: “Toward Automated Requirements Engineering: Empirical and Architectural Foundations for Structured Parsing and Knowledge Discovery”
Certifications
Teaching & Authored
Let's talk
I'm open to research collaboration, AI engineering work, and good conversations about NLP for software engineering. The fastest way to reach me is email.
Supplementary material
Appendix A
The thesis processing pipeline as an interactive node editor: inspect each stage's inputs, outputs and metrics, and explore parameter configurations.
Appendix B
The ablation study as a playground: toggle components on and off to see how each contributes to performance against the baseline.