How to think about AI application development / AI 應用開發的一些想法

Anton Troynikov, Founder of Chroma

出處: https://x.com/atroyn/status/1819395539156557892

網頁製作和翻譯: ihower with Claude

yesterday evening i gave a presentation to founders, investors, and the ai community at aixventureshq

on how to think about ai application development. it was well received so i'm going to reproduce it in full here on x the everything app (which is also now a slide deck app).

昨晚我在 aixventureshq 向創業者、投資者和 AI 社群發表了一場演講,

主題是如何思考 AI 應用程式開發。反應相當熱烈,所以我決定在這裡完整地重現演講內容。

  1. buzzwords are mind killers. you must empty your head of all buzzwords. the temptation with any new technology is to use existing concepts as crutches as much as possible, but this kills the creativity necessary to explore the full capabilities of this new technology. new abstractions which don't carry meaning are unnecessary / harmful:

    • 'rag' is meaningless as a concept. imagine calling a web app 'database augmented programming'.
    • 'agent' probably just means 'run an llm in a loop'

    phd-sounding terms to say simple concepts mostly let everyone know how cool you are for readng the arxiv but don't help anyone get work done:

    • 'prompting' means 'text input'
    • 'system prompt' means 'the instruction part of the text input'

    it is far too early to spend time adopting the abstractions imposed by existing tools and frameworks. until you understand what you want to build and how to build it, these can only slow you down

    finally, ai application development is still just programming, with the same social and technological implications as all programming since we invented the stored instruction computer

    流行詞彙是思維殺手。你必須清空腦中所有的流行詞彙。面對任何新技術時,我們都傾向盡可能地依賴現有概念,但這會扼殺探索這項新技術全部潛力所需的創造力。新的、沒有實質意義的抽象概念是不必要且有害的:

    • RAG(檢索增強生成)作為一個概念是毫無意義的。想像一下把網頁應用稱為「資料庫增強程式設計」。
    • Agent(代理)可能只是意味著「迭代呼叫 LLM」

    使用聽起來像博士論文的術語來描述簡單概念,主要是讓大家知道你有多酷,因為你讀過 arXiv,但這對實際工作毫無幫助:

    • Prompting(提示)意味著「文字輸入」
    • System prompt(系統提示)意味著「文字輸入中的指令部分」

    現在採用現有工具和框架所強加的抽象概念還為時過早。在你理解你想要建立什麼以及如何建立之前,這些只會拖慢你的腳步。

    最後,AI 應用程式開發仍然只是程式設計,與自從我們發明指令存儲電腦以來的所有程式設計一樣,具有相同的社會和技術含義。

  2. chatbots / assistants are the geocities (static websites if you're too young to remember the true glory days) of ai

    • people think 'ai' means 'chatbot' because their first experience of ai is chatgpt / claude / gemini etc, and because people reason about new technologies by analogy. this is the same principle as how at launch, the web was thought of as a new publishing medium (hence web 'page'). it takes time to break out of the very first analogies we apply.
    • chatbots and assistants are very useful, very flexible, and apply across almost every industry and job function. it's great to be able to interact with large text corpora non-linearly and conversationally, and 95% of chroma's users are building chatbots on top of their data. this is real and very useful and people should do that!
    • ... but they just represent static data in a new way. even assistants without retrieval are essentially a kind of conversational index with basic reasoning over a corpus stored in the weights of the model (this is too many phd words)
    • over-indexing on chatbots as the most important or most valuable application of this technology is a mistake, but so is dismissing their utility

    聊天機器人/助手是 AI 的 GeoCities(如果你太年輕不記得真正的輝煌時代,那就是靜態網站)

    • 人們認為「AI」意味著「聊天機器人」,因為他們第一次接觸 AI 是通過 ChatGPT/Claude/Gemini 等,而且人們總是通過類比來理解新技術。這與網絡剛推出時被認為是一種新的出版媒介(因此稱為網頁)的原理相同。擺脫我們應用的第一個類比需要時間。
    • 聊天機器人和助手非常有用、非常靈活,幾乎適用於每個行業和工作職能。能夠以非線性和對話方式與大型文本語料庫互動是很棒的,95% 的 Chroma 用戶都在他們的數據之上構建聊天機器人。這是真實的,非常有用,人們應該這樣做!
    • ...但它們只是以新的方式呈現靜態數據。即使是沒有檢索功能的助手,本質上也只是一種對存儲在模型權重中的語料庫進行基本推理的對話式索引(這裡用了太多博士級詞彙)
    • 過度關注聊天機器人作為這項技術最重要或最有價值的應用是一個錯誤,但忽視它們的實用性同樣是錯誤的
  3. what is an llm? as far as application development is concerned, an llm is;

    • an information processing system
    • that processes unstructured information
    • in a common sense way
    • that can be accessed over an api

    that's it. an llm is something you can integrate into software to build new kinds of programs. you absolutely do not need to know what a transfomer even is let alone how one works to build something.

    什麼是 LLM?就應用程式開發而言,LLM 是:

    • 一個資訊處理系統
    • 處理非結構化資訊
    • 以一種常識的方式
    • 可以通過 API 訪問

    就是這樣。LLM 是可以整合到軟體中以構建新型程式的東西。你絕對不需要知道 Transformer 是什麼,更不用說它是如何工作的,就能建立一些東西。

  4. llms as processing elements allow us to write new kinds of programs, which we call 'ai applications'. these are programs which would have been impossible to write before because we had no way of getting computers to use common sense to process unstructured information in the way humans might. now we do.

    作為處理元素的 LLM 允許我們編寫新型態的程式,我們稱之為「AI 應用程式」。這些是以前不可能編寫的程式,因為我們沒有辦法讓電腦像人類那樣用常識處理非結構化資訊。現在我們可以了。

  5. software and computing in general is for the automation of information processing tasks. most of the value in software comes from automating economically valuable activities. ai is for business process automation.

    一般來說,軟體和計算是用於自動化資訊處理任務。軟體的大部分價值來自於自動化經濟有價值的活動。AI 是用於業務流程自動化。

  6. what kinds of things can we build that are economically valuable? i took five minutes out of my day writing software to write down some stuff llms can do for me and calculated the economic value of these automations. you an probably save hundreds of thousands of dollars a year with a reasonably sized software engineering team, for the low cost of nothing. this analysis excludes the extra money your business can make by reducing cycle times / time to market, reducing the number of issues your team ships, etc.

    DynamicWebPaige has great demos that do similar things from the user research perspective. these are things you can have today; you don't need to wait for the models to become superintelligent / we approach the singularity event horizon. you don't have to think very hard about how to implement them either, these are just text processing tasks.

    我們可以建立哪些具有經濟價值的東西?我花了五分鐘時間寫下 LLM 可以為我做的一些事情,並計算了這些自動化的經濟價值。你可能可以為一個規模合理的軟體工程團隊每年節省數十萬美元,而且幾乎不需要成本。這個分析不包括通過減少週期時間/上市時間,減少團隊發布的問題等方式為你的業務帶來的額外收益。

    DynamicWebPaige 從用戶研究的角度提供了很棒的演示,展示了類似的東西。這些都是你現在就可以擁有的東西;你不需要等待模型變得超級智能/我們接近奇點事件地平線。你也不需要太費心思考如何實現它們,這些只是文本處理任務。

  7. none of this requires anything except hooking up some business logic to an llm api / or an llm running locally. you don't need aything but your text editor and an api key. stop over thinking this and start building. no thoughts head empty.

    所有這些都不需要任何東西,除了將一些業務邏輯連接到 LLM API / 或本地運行的 LLM。你只需要你的文本編輯器和一個 API 密鑰。停止過度思考,開始構建。不要想太多,保持頭腦清空。

  8. ai is about software, software is about automation, ai is about automation. ai lets us automate things we could not automate before. ai as an industry has spent the last 2.5 years marketing this technology as something that can write an essay or an email, instead of process information. calling any of this 'generative ai' has been incredibly damaging for helping people think about it, and unfortunately the first name that sticks tends to define a thing even if the name is bad.

    AI 是關於軟體的,軟體是關於自動化的,AI 是關於自動化的。AI 讓我們能夠自動化以前無法自動化的事情。AI 作為一個行業在過去 2.5 年裡一直在將這項技術宣傳為可以寫文章或電子郵件的東西,而不是處理資訊。將任何這些稱為「生成式 AI」對幫助人們思考它是極其有害的,不幸的是,第一個流行的名稱往往會定義一件事,即使這個名稱不好。

  9. just like every new kind of software, ai comes with a set of new techniques that we need to learn to use it effectively. we are still learning what those techniques are, but some are starting to emerge:

    • evals are like the unit / integration tests of ai applications, except they're not binary pass/fail but instead measure whether outputs are acceptable in a continuous rather than binary fashion. this is very hard to understand and get used to in software development, and yet we must.
    • retrieval (like chroma) is like the database component of the ai application stack. because llms process information in a common sense way, they must also address stored information in a semantic way - hence chroma. now, you can also use retrieval to store and retrieve instructions, which makes retrieval more like the memory and storage subsystem for llms as a computing primitive. but i'm ahead of myself, more on that at a different time.
    • ai applications are uniquely iterative in both the way they're built, and the way they're used. think about how you might use midjourney to refine an image generation that isn't quite right from the start, until it's what you want. because we cannot pre-define what output we expect from a given input, we find that we must iterate toward our goal and then make sure that the solution we converged to generalizes across our use-case.
    • ai uses common sense to process information which means you can also instruct it in natural language. this breaks down the abstraction barrier (learning to program) that would otherwise be prevent people from building useful tools for their jobs. virtually anyone can experiment with this stuff with minimal fuss.

    就像每一種新型軟體一樣,AI 帶來了一套我們需要學習才能有效使用它的新技術。我們仍在學習這些技術是什麼,但一些已經開始浮現:

    • 評估就像 AI 應用程式的單元/集成測試,只是它們不是二元的測試通過或失敗,而是以連續而非二元的方式衡量輸出是否可接受。在軟體開發中,這是非常難以理解和適應的,然而我們必須這樣做。
    • 檢索(如 Chroma)就像 AI 應用程式堆疊的資料庫組件。因為 LLM 以常識的方式處理資訊,它們也必須以語義的方式處理存儲的資訊 - 因此有了 Chroma。現在,你還可以使用檢索來存儲和檢索指令,這使得檢索更像是 LLM 作為計算原語的記憶體和存儲子系統。但我有點超前了,這個話題我們改天再談。
    • AI 應用程式在構建方式和使用方式上都是獨特的迭代性的。想想你可能如何使用 Midjourney 來改進一開始不太完美的圖像生成,直到它符合你的要求。因為我們無法預先定義我們期望從給定輸入得到什麼樣的輸出,我們發現我們必須朝著我們的目標迭代,然後確保我們收斂到的解決方案可以在我們的用例中普遍適用。
    • AI 使用常識來處理資訊,這意味著你也可以用自然語言來指導它。這打破了抽象障礙(學習程式設計),否則這種障礙會阻止人們為他們的工作建立有用的工具。幾乎任何人都可以用最少的麻煩來實驗這些東西。
  10. trychroma is for the builders.

    trychroma 是為建設者準備的。