Aaron Liang · Agentic AI Systems Engineering × EDA Automation

Building AI Agents That Actually Work.

從工具調度、長期記憶到故障恢復,我把 LLM 的推理能力轉化為可靠、可驗證的執行系統。以下是我一個人設計、實作,從 2026 年 6 月起 24/7 運轉至今的系統。

From tool orchestration and long-term memory to failure recovery, I turn LLM reasoning into reliable, verifiable execution. Below is a system I designed and built solo, running 24/7 since June 2026.

7,106則結構化知識筆記structured knowledge notes
99.8%筆記進入知識圖譜的覆蓋率knowledge-graph coverage
7–56 s → 0.9–1.6 s記憶搜尋時間,結果不變memory search time, identical results
看系統怎麼運作 ↓Explore the System ↓ 數據取自系統紀錄,截至 2026-10-13Figures from system logs, as of 2026-10-13

01 · In Action

From Information to Action.

一段會議錄音如何變成被執行、被記住的工作。每一步都標明是誰在做決定——固定排程不等於 Agent 自主決策。

How one meeting recording becomes work that gets done and remembered. Each step shows who decides — a fixed schedule is not agent autonomy.

DETERMINISTIC 固定程式與排程fixed code & schedules LLM WORKFLOW 固定流程中的 LLM 步驟an LLM step in a fixed pipeline AGENT DECISION Agent 自行選擇工具與下一步the agent picks tools & next step HUMAN APPROVAL 人按下核准才執行runs only after a human approves
  1. 擷取Capture

    DETERMINISTIC

    每 2 分鐘輪詢錄音雲端。還沒轉錄完、或 AI 摘要還沒出來的錄音進等待佇列退避重試,不拿半成品往下走。已處理 674 段錄音。

    Polls the recorder's cloud every 2 minutes. Recordings without a finished transcript or AI summary wait in a backoff queue instead of moving on half-done. 674 recordings processed.

  2. 理解Understand

    LLM WORKFLOW

    LLM 把逐字稿蒸餾成事實、決定、點子、任務等原子筆記,輸出受嚴格 JSON Schema 約束;人名與專案名透過別名表統一。

    An LLM distills the transcript into atomic notes — facts, decisions, ideas, tasks — under a strict JSON Schema; names are normalized through an alias table.

  3. 分派Route

    LLM WORKFLOW

    每個行動項標上「負責人 × 類型」。2,225 個行動項中有 62% 是別人的承諾——只按類型派工會派錯人,所以先判斷負責人。

    Every action item is labeled owner × type. Of 2,225 items, 62% were other people's commitments — routing by type alone would assign the wrong party, so ownership is decided first.

  4. 執行Act

    AGENT DECISION

    屬於 Agent 的研究任務交給沙箱中的研究 Agent,由它自行規劃搜尋與查證,已完成 28 個。有人提問時,總調度 Agent 從 18 個工具中自行選擇要查記憶、讀 Jira/Slack,或建立任務。

    Research tasks owned by the agent go to a sandboxed research agent that plans its own searches and verification — 28 completed. On request, an orchestrator agent chooses among 18 tools: recall memory, read Jira or Slack, or create work.

  5. 把關Govern

    HUMAN APPROVAL

    依風險分級:唯讀直接執行;只寫入自己記憶的可逆操作自動;團隊看得到的寫入一鍵核准;對外建立新物件每次都要人按核准,而且不提供「永遠允許」。

    Tiered by risk: reads run directly; reversible writes to private memory run automatically; team-visible writes need one-tap approval; creating anything external needs a human every time — "always allow" isn't offered.

  6. 回流與修復Learn & Recover

    DETERMINISTIC LLM WORKFLOW

    結果寫回檔案,自動進入知識圖譜,下次查得到。每 15 分鐘對帳一次,補做失敗或遺漏的項目。

    Results are written back to files and flow into the knowledge graph, retrievable next time. A reconciliation pass every 15 minutes redoes anything that failed or was missed.

真實執行紀錄:一個研究任務Real execution trace: one research task

2026-09-14,台北時間;內容為摘要,原文是內部筆記2026-09-14, Taipei time; summarized from internal notes
  1. 會議 → 任務Meeting → task會議提到要評估能分辨說話者的語音辨識服務,蒸餾後被分派為 actor=agent · route=research。In a meeting, the team agreed to evaluate speech recognition that can tell speakers apart. The note was distilled and routed as actor=agent · route=research.
  2. Agent 接手Agent starts研究 Agent 只拿到網路搜尋與讀取工具,沒有檔案、Shell 或任何寫入權限。The research agent gets web search and fetch only — no file, shell or write access.
  3. 查證與產出Verify & report完成報告,引用 33 個來源,比較 3 家服務的價格與語者分離能力。Report delivered with 33 sources, comparing pricing and diarization across 3 services.
  4. 回流Write back由排程流程(不是 Agent)把報告寫回知識庫,以固定流程中的 LLM 步驟萃取成可重用的 playbook,關閉任務並附上連結,摘要推送到手機。A scheduled pipeline, not the agent, writes the report into memory, has a fixed LLM step distill a reusable playbook, closes the task with a link and pushes a summary to my phone.

關鍵在它查不到的部分:會議裡講的兩個產品名稱查無此物。Agent 沒有硬湊答案,而是指出很可能是語音轉錄聽錯、提出最可能的候選並標記「未經證實」,列為第一個待確認事項。

The telling part is what it couldn't find: two product names from the meeting don't exist. Instead of inventing answers, the agent flagged a likely transcription error, proposed the most probable candidates marked unverified, and made confirming them the first action item.

02 · Architecture

The LLM Reasons. The System Keeps It Reliable.

讓 AI 負責推理,讓工程系統確保可靠。實線是資料流,虛線是控制與回饋。

AI does the reasoning; engineering makes it reliable. Solid lines carry data; dashed lines carry control and feedback.

INPUTSORCHESTRATIONAGENTS & WORKFLOWSMEMORY & TOOLS 會議錄音Meeting audio Email · 行事曆Email · calendar 我的提問(手機)My request (phone) 排程管線Scheduled pipelines 22 個常駐服務 · launchd22 always-on services · launchd 輪詢 · 退避 · 等待上游polling · backoff · wait upstream DETERMINISTIC 總調度 AgentOrchestrator agent 理解意圖 · 從 18 個工具中選擇reads intent · picks from 18 tools AGENT DECISION 蒸餾:逐字稿 → 原子筆記Distiller: transcript → notes 分派:負責人 × 類型Router: owner × type 圖譜萃取:實體與事實Graph extraction: entities & facts 研究 Agent(沙箱)Research agent (sandboxed) 只能搜尋與讀網頁,不能寫入web search & fetch only; cannot write 長期記憶Persistent memory Markdown 檔案=唯一事實來源Markdown files = source of truth 時序知識圖譜(可丟棄重建)temporal knowledge graph (rebuildable) 12,188 條事實 · 3,376 個實體12,188 facts · 3,376 entities 受限記憶 API(token 驗證)Restricted memory API (token) 外部工具(白名單)External tools (allowlisted) Jira · Confluence · Slack · Web agent 任務 ↓agent tasks ↓ GOVERNANCE & RELIABILITY 風險分級閘門Risk-tiered gate 寫入在程式層攔截,人核准才呼叫 writes intercepted in code;called only after approval 工具呼叫驗證Tool-call verification 宣稱的動作必須對得上實際呼叫 claimed actions must matchthe real tool calls 對帳 · 退避 · 安全閥Reconcile · backoff · valve 每 15 分鐘補做;異常過多就停手 redo every 15 min;halt if anomalies spike 隔離與成本記帳Isolation & cost ledger 對外 Agent 在獨立 VM;每次 LLM 呼叫都入帳 outward-facing agent in its own VM;every LLM call metered 結果回寫記憶 ↑results written back to memory ↑

← 左右滑動看完整架構 →← swipe to see the full diagram →

約束寫在程式,不寫在 prompt

Constraints live in code, not prompts

Agent 碰不到執行點。prompt 裡的規則是建議,程式裡的閘門才是約束。

The agent never touches the execution point. A rule in a prompt is a suggestion; a gate in code is a constraint.

自主權用紀錄換

Autonomy is earned

新類型的任務先只提案、不執行,驗證過品質才開放自動執行。

New task types start as proposals only and run automatically once their quality is proven.

Technical deep dive

安全模型:假設任何一層都會被繞過Security model: assume any single layer gets bypassed
  • 對外的總調度 Agent 跑在獨立的沙箱 VM,只能透過 token 驗證的受限 API 碰記憶。
  • 總調度 Agent 可呼叫的外部系統走白名單(Jira、Confluence、Slack),其餘一律拒絕;研究 Agent 只有網路搜尋與讀取。
  • Slack、Jira、網頁內容一律標記為「資料」而非「指令」,抵擋 prompt injection;讀到可疑指令只回報、不執行。
  • 研究 Agent 的子程序剝除所有機密環境變數,prompt 走 stdin,不繼承任何使用者層的工具設定。
  • The outward-facing orchestrator runs in its own sandboxed VM and reaches memory only through a token-authenticated API.
  • The orchestrator's external systems are allowlisted (Jira, Confluence, Slack); everything else is refused. The research agent gets web search and fetch only.
  • Content from Slack, Jira or the web is tagged as data, never instructions, to resist prompt injection — suspicious instructions are reported, not executed.
  • The research agent's subprocess is stripped of every secret env var, receives its prompt over stdin and inherits no user-level tool config.
成本治理:每一次 LLM 呼叫都入帳Cost governance: every LLM call is metered
  • 知識萃取每則筆記約 6 次 LLM 呼叫,是最大的成本來源。chat、structured output、embedding 三個出口統一記帳,按子系統拆分用量,另有服務監看額度。
  • 一次設定錯誤曾讓萃取改走高價模型、單價高出 21 倍;之後模型的設定值與實際生效值都必須可觀測。
  • 選模型實測過限制:推理型模型會把輸出額度吃光、回傳空白,因此萃取不用推理型模型。
  • Knowledge extraction costs about six LLM calls per note — the largest expense. All three exits (chat, structured output, embeddings) are metered and split by subsystem, and a separate service watches the budget.
  • One misconfiguration once routed extraction to a model 21× pricier; since then the configured and the effective model must both be observable.
  • Model limits were tested, not assumed: reasoning models can burn the whole output budget and return nothing, so extraction avoids them.
我每天在用的六套自動化(多數是排程流程,不是 Agent)Six automations I run every day (mostly scheduled pipelines, not agents)
  • 第二大腦Second brainDETERMINISTIC LLM WORKFLOW

    watcher 偵測每一則新筆記,經 LLM 分類歸檔,再由知識圖譜把跨筆記的人事物串起來,每晚自動 commit 備份。A watcher picks up every new note, an LLM pass files it, and a knowledge graph links people and topics across notes; committed and backed up nightly. 每天都在用daily driver

  • 會議 → 筆記 → 任務Meeting → notes → tasksDETERMINISTIC LLM WORKFLOW

    自動取回錄音與摘要,蒸餾成決策/事實/任務,再把任務同步到看板(流程細節見上方「In Action」)。Pulls recordings and summaries, distills decisions, facts and tasks, and syncs tasks to a board (see “In Action” above for the flow). 每場會議後自動跑after every meeting

  • Donna:架在大腦前面的對話助理Donna, a chat assistant over the brainAGENT DECISION HUMAN APPROVAL

    Telegram 上的助理:問它當時決定了什麼、叫它記一件事、讓它派工。寫入依風險分級:只動我自己記憶的直接寫入,團隊看得到或對外的才等我核准。A Telegram assistant: ask what was decided, tell it to record something, have it dispatch work. Writes follow the risk tiers: private memory writes go straight through; team-visible or external ones wait for my approval. 我的手機端介面my phone-side interface

  • 每日教練Daily coachDETERMINISTIC LLM WORKFLOW

    讀我的三層目標(方向/年度/季度下注)與偏好,每天早上給一個具體建議。Reads my three tiers of goals (direction / annual / quarterly bets) and preferences, and sends one concrete suggestion each morning. 每天早上every morning

  • 短影音生產線Short-form video pipelineDETERMINISTIC

    手機素材進去,出來逐字稿、剪點提案、燒好的字幕與程式合成的配樂,最後自動驗收長度、色彩與響度。Raw phone footage in; transcript, proposed cuts, burned-in subtitles and code-synthesized music out, then automated QC on length, color and loudness. 已產出六支six videos shipped

  • 社群成長追蹤+每日簡報Social growth tracking + daily briefDETERMINISTIC LLM WORKFLOW

    每天把 Instagram 指標快照存進版控的 CSV,早上推一份簡報:今天發什麼、拍什麼、哪些留言還沒回。Daily Instagram metric snapshots into version-controlled CSVs, plus a morning brief: what to post, what to shoot, which comments still need replies. 2026-08 建置built 2026-08

實際使用的技術What it actually runs on

Python 與 launchd 排程;Claude、GPT、Gemini 經 OpenRouter 呼叫;時序知識圖譜 Graphiti + FalkorDB;Markdown + Obsidian 作為檔案層;MCP 橋接 Jira/Confluence/Slack;Telegram 作為手機介面與核准按鈕。

Python with launchd scheduling; Claude, GPT and Gemini via OpenRouter; Graphiti + FalkorDB for the temporal knowledge graph; Markdown + Obsidian as the file layer; MCP bridges to Jira, Confluence and Slack; Telegram as the mobile interface and approval surface.

03 · Engineering Proof

Three Failures and How I Fixed Them

工程能力要看系統出問題之後怎麼處理。每個數字都來自系統自己的紀錄。

Engineering shows after things break. Every number comes from the system's own logs.

Performance

7–56 s → 0.9–1.6 s

記憶搜尋最佳化Memory search optimization

Problem
圖譜長到上萬條關係後,搜尋開始逾時,連帶卡住寫入。Past ten thousand relations, search began timing out and stalling writes.
Decision
追到資料庫的查詢計畫:全文搜尋的每個命中都重新掃描全部實體,成本是「命中數 × 實體數」。改成直接取用命中關係的兩端節點。Traced the database query plan: every full-text hit rescanned all entities — cost hits × entities. Switched to reading each hit's endpoints directly.
Result
同一批查詢、結果完全相同,時間降到 0.9–1.6 秒。Same queries, identical results, 0.9–1.6 s.
驗證方式How it was verified
  • 先以唯讀方式在正式圖譜上比對新舊寫法,回傳的關係、節點與分數逐筆相同。
  • 寫入流程在隔離的臨時資料庫驗證(寫入、搜尋、刪除),通過後才上線。
  • 上線後連續萃取未再出現逾時。
  • Compared old and new queries read-only on the production graph: returned relations, nodes and scores matched row for row.
  • Validated the write path (add, search, delete) in an isolated throwaway database before shipping.
  • No timeouts in consecutive extractions after release.
Reliability

987

找回靜默遺失的記憶Silently lost memories recovered

Problem
對帳發現圖譜比原始筆記少了 987 則,而且沒有任何錯誤訊息。Reconciliation showed the graph 987 notes short of the source files — with no error raised.
Decision
拆出三個成因:逾時後不再重試、批次期間新進的檔案被漏掉、狀態記為完成但資料已遺失。三種都加上自動偵測與補做。Isolated three causes: timeouts never retried, files arriving mid-batch skipped, and state marked done for data that was gone. Added detection and repair for all three.
Result
修正後連續 862 / 862 次萃取成功、0 失敗,覆蓋率到 99.8%。862 / 862 consecutive extractions succeeded after the fix, 0 failures; coverage reached 99.8%.
修復機制Repair mechanism
  • 每 15 分鐘掃描「該完成卻沒完成」的項目,逐一補做,寫入一次只處理一則。
  • 失敗以 10 分鐘起、上限 6 小時指數退避:不會放棄重試,也不會密集轟炸。
  • 安全閥:對不上的比例超過 20%,判定為連錯資料庫或資料被清空,停手報警,不盲目重跑上萬次 LLM 呼叫。
  • 以 11 項自動化測試覆蓋三種成因與安全閥。
  • Every 15 minutes, find work that should be done but isn't and redo it — one write at a time.
  • Failures back off exponentially from 10 minutes to a 6-hour cap: never gives up, never hammers.
  • Safety valve: if more than 20% of records mismatch, it assumes a wrong database or wiped data, stops and alerts instead of blindly re-running tens of thousands of LLM calls.
  • 11 automated tests cover all three causes and the valve.
Agent Reliability

驗證它做了什麼Verify what it did

工具呼叫驗證Tool-call verification

Problem
Agent 回答「我搜尋了所有記憶,沒有找到」,但服務端紀錄顯示那段時間一次搜尋請求都沒有。The agent replied "I searched all your memories and found nothing" — server logs showed zero search requests.
Decision
不再只靠 prompt。在程式層比對回覆內容與實際的工具呼叫:宣稱查過卻沒呼叫,就退回重做,且假回覆不寫進對話歷史。Stopped relying on the prompt. Code now compares the reply against the actual tool calls: a claimed search with no call is sent back, and the false reply never enters history.
Result
15 項重現真實事件的自動化測試全數通過後上線;同樣的問題現在會真的去查。Shipped after 15 automated tests reproducing the real incident passed; the same question now triggers a real lookup.

其他上線過的修正:錄音在 AI 摘要產生前就被處理而漏掉重點,改為等待上游,補回 4 段長錄音、38 則筆記;背景 Agent 改名後被誤判掉線、重複開了兩次空白 session,改以 session 身分判斷。

Other shipped fixes: recordings processed before their AI summary existed lost key points — now the pipeline waits on upstream, recovering 4 long recordings and 38 notes; background agents misread as offline after a rename spawned two duplicate blank sessions — now identified by session, not name.

04 · Beyond Agents

Where the Same Approach Applies

同一套系統工程能力,跨越不同的專業領域。

The same systems-engineering discipline, applied across domains.

邁向可驗證的 Agentic EDAToward Verifiable Agentic EDA

PROVEN

陽明交大電機與電腦工程博士。研究標準元件 layout 自動生成、擺置與繞線、以機器學習處理設計規則;我在聯發科開發的 AI 元件 layout 生成方法與流程,已導入聯發科產線。

Ph.D. in Electrical & Computer Engineering (NYCU). Research on automated standard-cell layout, placement and routing, and machine learning for design rules. The AI cell-layout generation methods and flow I developed at MediaTek were adopted in its production line.

下一步 · 研究方向NEXT · RESEARCH DIRECTION
  • LLM 讀電路圖與連線推理的能力基準與幻覺率
  • 以可驗證為前提的 Agentic EDA 編排
  • Benchmarks for LLM schematic reading, connectivity reasoning and hallucination rate
  • Verifiable agentic orchestration of EDA flows

這是研究方向,尚未建成系統。

A research direction — not a built system yet.

看半導體研究脈絡 →See the semiconductor research →

把 LLM 放進可靠的系統LLMs Inside Reliable Systems

把 LLM 放進有記憶、有權限、會被驗證、出錯能恢復的系統裡——上面的架構與三個工程案例就是證據。

Putting an LLM inside a system with memory, permissions, verification and recovery — the architecture and the three cases above are the evidence.

回到工程案例 ↑Back to the cases ↑

流程自動化,關鍵步驟由人核准Workflow Automation with Human Approval

同一套做法可以接到企業既有的資料與工具:會議、信件、專案系統。可逆的事自動做,對外的事等人核准,失敗由定期對帳補做,異常太多時停手通知人。

The same approach connects to a company's existing data and tools — meetings, email, project systems. Reversible work runs automatically, anything external waits for a human, and a reconciliation pass redoes failed work — stopping to alert me when too much looks wrong.

對照 IC 設計流程的 Super Flow 分層Mapping to an IC design Super Flow
層級Layer在這套系統裡In this system在 IC 設計流程裡In an IC design flow
Subflow單一職責的 Agent/流程,輸出受 schema 約束single-purpose agents and workflows with schema-bound output例:標準元件生成,有明確的輸入輸出契約e.g. standard-cell generation with a clear I/O contract
Main Flow冪等、退避重試、定期對帳的管線idempotent pipeline with backoff and reconciliation長而昂貴的設計步驟,失敗要能定位與續跑long, expensive design steps that must resume, not restart
Super Flow總調度 Agent + 風險閘門 + 結果回流orchestrator agent + risk gate + feedback into memory選擇並串接子流程,自主等級逐級提升,人在關鍵點簽核select and chain subflows, raise autonomy level by level, human sign-off at key points

想打造可靠的 AI 系統?歡迎聯絡。Interested in building reliable AI systems? Let's connect.