FilingLens: Evidence-Grounded RAG Agent

FilingLens is an Agentic RAG system for selected U.S. public-company analysis. It turns natural-language finance questions into controlled research workflows over SEC filings, structured financial facts, derived metrics, and auditable answer traces.

View Demo GitHub

Overview

FilingLens turns financial analysis into a traceable, evidence-grounded workflow. Instead of letting the LLM answer directly, it plans evidence, retrieves filings, computes metrics, and validates the final response before presenting it.

Evidence-backed answers

Answers are grounded in financial facts, filing citations, limitation notes, and trace records.

SEC filing retrieval

10-K and 10-Q filings are cleaned, sectioned, embedded, and retrieved with company, form, period, and section metadata.

Deterministic metrics

Derived metrics are computed by code from structured financial facts before drafting.

Auditable workflow

The agent records intent, evidence plans, tool calls, sufficiency checks, and answer-contract validation.

Problem

Financial analysis questions are rarely solved by text retrieval alone. They require numeric facts, filing context, comparative reasoning, and clear risk boundaries. Generic LLM answers may sound fluent but remain unsupported, while basic RAG can retrieve relevant passages without proving the full analytical claim.

Approach Strength Where It Breaks FilingLens Response
Generic LLM chat Produces fluent financial commentary. May invent numbers, miss filing context, or cross into investment advice. Enforces evidence policy, deterministic metrics, and answer-contract checks.
Basic RAG Retrieves related filing passages. Often lacks structured facts, computed metrics, and evidence sufficiency status. Combines SEC filing retrieval, DuckDB facts, metric calculators, and cited evidence packets.
FilingLens Runs a bounded research workflow. Still depends on filing coverage, structured data, and retrieval quality. Degrades, caveats, repairs, or blocks answers when evidence is insufficient.

Architecture

FilingLens separates evidence collection, metric computation, and validation from LLM-generated synthesis. Each response moves through a controlled, auditable research workflow before it is returned.

Backend: FastAPI. Orchestration: LangGraph. Retrieval: ChromaDB. Structured facts: DuckDB. Models: local vLLM / Qwen3-8B-AWQ or hosted OpenAI-compatible APIs.

RAG Pipeline

Raw SEC filings are converted into section-aware, metadata-rich evidence chunks before being used by the agent for retrieval, citation, and validation.

Raw Filings Main Document Clean Text Section Chunks Vector Index Cited Evidence
  1. Download filingsCollect 10-K and 10-Q filings for tracked companies.
  2. Extract main documentIdentify the primary filing document before cleaning and chunking.
  3. Clean HTML / XBRLRemove markup, boilerplate, table noise, and non-analytical fragments.
  4. Create section-aware chunksPreserve Item, section, ticker, form, and period metadata for filtering and citation context.
  5. Build vector indexEmbed filing chunks and persist them in Chroma with structured metadata.
  6. Filter and rerank evidenceApply company, form, period, and section constraints before selecting cited evidence.

Agent Workflow

FilingLens treats open-ended financial analysis questions as bounded research tasks. Each query is converted into an intent, evidence plan, tool path, sufficiency check, and answer contract before drafting begins.

Example question

"Why did operating margin change over the last two fiscal years?"

  1. Understand the question Resolve the company, period, metric, comparison target, and safety boundary.
  2. Plan required evidence Specify the needed filing sections, structured facts, computed metrics, and citation requirements.
  3. Execute tools Run SEC filing retrieval, DuckDB fact queries, and metric calculators through constrained tool specs.
  4. Check sufficiency Label each evidence dimension as supported, partial, missing, or blocked.
  5. Draft from evidence Generate the answer only from the accepted evidence packet.
  6. Validate the contract Check citations, numbers, limitations, unsupported claims, and advice boundaries.

Challenges

Challenge Solution Engineering skill
Number hallucination Compute metrics from structured facts before answer drafting. Data modeling, deterministic computation, validation contracts
Noisy filing text Parse primary filing documents, clean HTML/XBRL noise, and preserve section metadata. Document processing, retrieval design, metadata strategy
Unstable tool calls Constrain agent steps with canonical intents, evidence policies, and typed tool specs. Agent orchestration, typed state, LangGraph workflow design
Insufficient evidence Use dimension status and answer contracts to add caveats, repair, downgrade, or block unsupported answers. Failure mode design, safety boundaries, auditability

Evaluation

The project includes an internal evaluation set for grounding, citation validity, status consistency, advice-boundary violations, and comparison balance. These checks serve as regression signals, not as a guarantee that every generated answer is correct.

Numeric groundingChecks whether financial figures are backed by structured facts or computed metrics.
Citation validityChecks whether cited filing snippets exist and support the related claim.
Status consistencyChecks whether supported, partial, missing, and blocked evidence statuses appear correctly in the final answer.
Advice-boundary checkChecks for buy, sell, hold, target-price, and personalized-advice violations.

Tech Stack

Frontend
HTML, CSS, JavaScript
Backend
FastAPI, Python, Pydantic
Agent / RAG
LangGraph, LangChain, ChromaDB, DuckDB
Models
OpenAI-compatible API, local vLLM / Qwen3-8B-AWQ, hosted model profiles, SentenceTransformers
Evaluation
Pytest, regression tests, answer benchmark, evidence contract checks

Roadmap

  • Expand retrieval benchmarks and automated regression evaluation.
  • Add news, events, and macro tools for richer context.
  • Improve trace visualization for debugging and production review.