> 歡迎訂閱我的 [AI Engineer 電子報](https://aihao.eo.page/6tcs9) 和瀏覽 [[Generative AI Engineer 知識庫]]
課程介紹網址: https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development
由 LangChain 作者 Harrison Chase 作者親自教，帶著看範例，的確比直接看 LangChain 文件好入門多了。
因為我上個月已經把 langchain 官方文件幾乎看過一輪了，所以內容偏簡單，我不很認真的看完。而且講者表達的不是很好，到底是講啥還不如直接看範例 code 就懂了。
主要值得一提的是 evaluation 那章有兩個新資訊(不在官方文件上!):
1. 有個全域的 debug 模式，可以印出到底 LangChain 發出什麼 prompts 出去:
import langchain
langchain.debug = True
這個重要的 debug mode 功能，翻遍官方文件都沒寫 (效果跟 verbose=True 不一樣，這更完整)
發現是作者 [https://twitter.com/hwchase17/status/1658886069725831170](https://l.facebook.com/l.php?u=https%3A%2F%2Ftwitter.com%2Fhwchase17%2Fstatus%2F1658886069725831170%3Ffbclid%3DIwAR08LsJYqTiDhCvNCSChsDfrHD9Z1EeO_c2IGM6j7fJfh4AZuDCmr2Kzdz4&h=AT2cTIVAkqaW138VPAihUuwwr4H2oC99IAHivb6gVewrPNRXV_-_o9Gunl-0nKmgodIq_9hLFsWL9qjlD9Dto-xciK--vV7Vj8DXKNqOLVhdpUdswXsEx1dfg5QI87nkKUcHnSs&__tn__=-UK-R&c[0]=AT2XY0WfVQ1NXU1C9nTQ2DTcgvAc7AtYB4BvWceHRVRZ2WIj8jbDf1fXS_GW6SMzP5EYh17pDKTEaD8NU8rolbPxIUxayiOCdVuKPvvBZRxihuqiL6CGTeVMnPSJ16uUe7elWw-8vmRzs9STAeuDGGZqa2WxhAC1CMECtfVpwmymYp1WjdTCPsPWOUHVitWplMnxz41zEVf8igA) 在 5/18 才加的新功能。
2. 冒出一個 LangChain plus 的 Web 介面評估平台 (evaluation platform):
這可以方便做實驗紀錄和評估。但講者只有展示畫面，卻沒有告訴我們怎樣可以安裝使用?? 難道是 LangChain 的付費功能? 再次找 source code 挖到了，這功能請去 [https://www.langchain.plus/](https://l.facebook.com/l.php?u=https%3A%2F%2Fwww.langchain.plus%2F%3Ffbclid%3DIwAR08LsJYqTiDhCvNCSChsDfrHD9Z1EeO_c2IGM6j7fJfh4AZuDCmr2Kzdz4&h=AT3se9-AFt12VeInQrav_vvsa5U0zgBDUAYz5slEt4cQGCb6CLykokRjWS5gMoBlXg5gO5_tPhFtw5efcowAv-qD_yOJ4G3shpydPm283fh0ubemRztKmr2XHXtbQP-lhr2129c&__tn__=-UK-R&c[0]=AT2XY0WfVQ1NXU1C9nTQ2DTcgvAc7AtYB4BvWceHRVRZ2WIj8jbDf1fXS_GW6SMzP5EYh17pDKTEaD8NU8rolbPxIUxayiOCdVuKPvvBZRxihuqiL6CGTeVMnPSJ16uUe7elWw-8vmRzs9STAeuDGGZqa2WxhAC1CMECtfVpwmymYp1WjdTCPsPWOUHVitWplMnxz41zEVf8igA) (目前看起來是免費使用，我猜之後會收費吧)
> (2023/7) 已改名 LangSmith 正式發布了: https://www.langchain.com/langsmith
> (2023/9) 這門課有接續的 RAG 課程 [[LangChain Chat with Your Data]]
## 1. Introduction
https://learn.deeplearning.ai/langchain/lesson/1/introduction
https://www.youtube.com/watch?v=gUcYC0Iuw2g&list=PLiuLMb-dLdWIYYBF3k5JI_6Od593EIuEG&index=1
![[Pasted image 20230609003626.png]]
![[Pasted image 20230609003640.png]]
![[Pasted image 20230609003653.png]]
## 2. Models, Prompts and parsers
https://learn.deeplearning.ai/langchain/lesson/2/models,-prompts-and-parsers
https://www.youtube.com/watch?v=VxAh0F08c9A&list=PLiuLMb-dLdWIYYBF3k5JI_6Od593EIuEG&index=2
* Parser 是指接收 model 的輸出，將輸出解析成結構化格式，以便後續處理
![[Pasted image 20230609013721.png]]
Prompt template
![[Pasted image 20230609013800.png]]
![[Pasted image 20230609013811.png]]
![[Pasted image 20230609013830.png]]
![[Pasted image 20230609013842.png]]
Output Parsers
![[Pasted image 20230609014232.png]]
![[Pasted image 20230609014427.png]]
這會錯，因為這是str
我們需要把回傳的 JSON str 轉成 python Dict
透過 ResponseSchema 和 StructuredOutputParser 來解析
![[Pasted image 20230609014449.png]]
![[Pasted image 20230609014529.png]]
![[Pasted image 20230609014559.png]]
## 3. Memory
https://learn.deeplearning.ai/langchain/lesson/3/memory
https://www.youtube.com/watch?v=rUPSs1lVvl4&list=PLiuLMb-dLdWIYYBF3k5JI_6Od593EIuEG&index=3
讓例如 Chatbot 的應用可以記住對話
* ConversationBufferMemory
* ConversationBufferWindowMemory
* ConversationTokenBufferMemory
* ConversationSummaryMemory
![[Pasted image 20230609033941.png]]
![[Pasted image 20230609033952.png]]
![[Pasted image 20230609034000.png]]
![[Pasted image 20230609033102.png]]
![[Pasted image 20230609033316.png]]
![[Pasted image 20230609033410.png]]
## 4. Chains
https://learn.deeplearning.ai/langchain/lesson/4/chains
https://www.youtube.com/watch?v=vLmK_mXuAsQ&list=PLiuLMb-dLdWIYYBF3k5JI_6Od593EIuEG&index=4
- LLMChain
- Sequential Chains
- SimpleSequentialChain
- SequentialChain
- Router Chain
![[Pasted image 20230609173233.png]]
![[Pasted image 20230609173204.png]]
![[Pasted image 20230609173244.png]]
![[Pasted image 20230609173531.png]]
![[Pasted image 20230609173320.png]]
![[Pasted image 20230609173337.png]]
![[Pasted image 20230609173546.png]]
![[Pasted image 20230609173943.png]]
![[Pasted image 20230609173357.png]]
![[Pasted image 20230609173409.png]]
![[Pasted image 20230609173424.png]]
## 5 .Question and Answer
https://learn.deeplearning.ai/langchain/lesson/5/question-and-answer
https://www.youtube.com/watch?v=gUcYC0Iuw2g&list=PLiuLMb-dLdWIYYBF3k5JI_6Od593EIuEG
![[Pasted image 20230611211252.png]]
![[Pasted image 20230611211304.png]]
![[Pasted image 20230611211335.png]]
![[Pasted image 20230611211503.png]]
![[Pasted image 20230611212118.png]]
![[Pasted image 20230611211411.png]]
這些用法在 summarization 場景上也有
![[Pasted image 20230611211602.png]]
![[Pasted image 20230611211611.png]]
> VectorstoreIndexCreator 用意未明，這個 wrapper 要幹嘛有點多餘。
## 6. Evaluation
https://learn.deeplearning.ai/langchain/lesson/6/evaluation
https://www.youtube.com/watch?v=2D8n5DNSmXI&list=PLiuLMb-dLdWIYYBF3k5JI_6Od593EIuEG&index=6
如何評估表現? 特別是當你換實作、改參數時。
要用什麼 data points 來評估?
1. 用自己準備好的 examples
2. 或用 QAGenerateChain 來自動產生問答 examples
![[Pasted image 20230612230540.png]]
![[Pasted image 20230612230550.png]]
如何評估?
import langchain
langchain.debug = True
![[Pasted image 20230612230710.png]]
debug = True 可以看到 retriever 階段找到的 context
QA 的結果有問題，常常是因為 retriever 階段找到的 context 不好
debug= True 也可以看到實際的 prompts 和 tokens 用量
> https://twitter.com/hwchase17/status/1658886069725831170 debug mode 是 5/18 才加的功能啊
批次跑出結果，然後用 QAEvalChain 來跑評估
![[Pasted image 20230612231408.png]]
最後介紹 LangChain 的評估平台 (evaluation platform)
![[Pasted image 20230612232526.png]]
![[Pasted image 20230612232535.png]]
![[Pasted image 20230612232544.png]]
![[Pasted image 20230612232607.png]]
![[Pasted image 20230612232628.png]]
> 但這個 LangChain Plus 課程裡面沒講去哪裡註冊使用，官方文件也沒寫。找 source code 之後發現可以直接去 https://www.langchain.plus 註冊，目前還是免費使用。我猜之後會收費吧。
## 7. Agents
https://learn.deeplearning.ai/langchain/lesson/7/agents
https://www.youtube.com/watch?v=FKE8XtrI5wc&list=PLiuLMb-dLdWIYYBF3k5JI_6Od593EIuEG&index=8
把 LLM 當作 推理引擎
這個場景會把 temperature 用 0
範例用 llm-math 跟 wikipedia tools
handle_parsing_errros = True 當LLM輸出有問題時，會自動重試
![[Pasted image 20230613175455.png]]
範例 PythonREPLTool，使用 create_python_agent 方法
![[Pasted image 20230613175510.png]]
定義你自己的工具
![[Pasted image 20230613175524.png]]
## 8. Conclusion
https://learn.deeplearning.ai/langchain/lesson/8/conclusion
https://www.youtube.com/watch?v=HeDjOjfiaW0&list=PLiuLMb-dLdWIYYBF3k5JI_6Od593EIuEG&index=8
## 網友心得
* https://twitter.com/dotey/status/1666503614720778259