Back to projects
Autonomous AI Agent for Network Design Review
Abstract.AI review of network design documents against live device configurations, owned from requirements to operation.
- Role
- AI Engineer, end-to-end owner
- Period
- Apr to Jul 2026
- Context
- ICT systems integrator (via Human Resocia)
- Team
- About 10
| 18/19 | Agreement with reviewer-judged findings; recurring false-positive classes eliminated |
| 25 → 10 | LLM calls per review job |
| min → sec | Re-processing time with the content-hash cache |
| Offline | Runs on air-gapped, CPU-only servers |
01From problem to approach
The problem
Design reviews meant checking long, multi-format documents against device configs by hand. LLM findings kept repeating the same classes of false positives, and the servers were air-gapped with no GPU.
The approach
Rebuild the fixed pipeline as an agent that plans one step at a time, verify every finding against the source text in code, and run the whole stack offline.
02How it works
- Input Design docs + configs Design documents (PDF, PPTX, DOCX, XLSX) and live device configurations.
- Step 02 Extract & selective OCR Text, tables, VLAN legends and topology; OCR only on low-quality pages.
- Step 03 Offline hybrid RAG BM25 + dense retrieval over ChromaDB, scoped to each device's hostname.
- Step 04 Agent plans & acts An LLM planner picks the next tool from a typed registry: decide, act, observe.
- Step 05 Guardrails & grounding Code-enforced checks, then verbatim verification of every finding against the source.
- Output Review report Grounded discrepancies with evidence, with live progress in Japanese.
03What I built
- Re-architected a fixed 7-stage LLM pipeline into an autonomous agent: an LLM planner selects tools step by step (decide, act, observe) from a typed tool registry.
- Enforced guardrails in code (preconditions, step budgets, a mandatory re-check and a deterministic fallback plan), with graceful degradation when a backend fails.
- Built a deterministic grounding layer over LLM findings: verbatim source verification with NFKC / full-width normalization, notation-system rules and section-label device attribution.
- Built a multi-format extraction pipeline for PDF, PPTX, DOCX and XLSX, with selective OCR on low-quality pages (Tesseract + OpenCV preprocessing) and python-pptx extraction of tables, VLAN legends and network topology.
- Engineered a fully offline RAG stack for air-gapped, CPU-only servers: multilingual sentence-transformers, ChromaDB, hybrid BM25 + dense retrieval, and hostname-scoped filtering to stop cross-device context leakage.
- Cut LLM calls per job from about 25 to 10 and added 3-level JSON self-healing; a content-hash cache cut re-processing from minutes to seconds.
- Root-caused production asyncio failures (semaphores bound to a stale event loop, dead executors inherited through contextvars) and fixed them at 4 layers.
- Served two backends, Gemini / Claude in the cloud and Ollama / Qwen2.5-7B locally, with live Japanese progress updates over WebSockets.
04Technologies
- Python
- Django · ASGI
- LangChain
- ChromaDB
- sentence-transformers
- BM25
- PyMuPDF
- python-pptx
- Tesseract OCR
- OpenCV
- PostgreSQL
- Gemini
- Claude
- Ollama · Qwen2.5
- WebSockets
- Docker