AI Engineer Summit 2023 - ihower's Notes

> 歡迎訂閱我的 [AI Engineer 電子報](https://aihao.eo.page/6tcs9) 和瀏覽 [[Generative AI Engineer 知識庫]] * Schedule: https://www.ai.engineer/summit/schedule * 10/9 直播錄影 https://www.youtube.com/watch?v=veShHxQYPzo * 10/10 直播錄影 https://www.youtube.com/watch?v=qw4PrtyvJI0 * 這人的筆記不錯 * 10/9 https://twitter.com/charlierguo/status/1711485076599644287 * 10/10 https://twitter.com/charlierguo/status/1711785345053065229 * 10/8 https://twitter.com/charlierguo/status/1711397011893997772 * AI Engineer Summit 2023 Recap https://kristinamacekovic.github.io/ai-engineer-summit-recap.html (2023/10/17) * 資料整理 https://twitter.com/neznam_neznam_n/status/1714701281824997409 (2023/10/19) 這場 Conference 的定位是應用層面的軟體工程師，因此講者跟題目，和機器學習類型的研討會很不一樣。沒有講 Transformer 深度學習模型、沒有講如何訓練模型、沒有講各種最新的 LLM 模型等，投影片也很少提到 papers。而是更多談開發框架議題、Prompt Engineering、RAG、Agents、有趣的應用、UX 等等。我推薦這幾場，內容詳見 https://ihower.tw/blog/archives/11794 > 以下截圖不是完整的，有興趣的話請去看完整錄影 # 10/9 ## swyx: The 1000x AI Engineer ![[Pasted image 20231010061908.png]] https://www.latent.space/p/ai-engineer ![[Pasted image 20231010061958.png]] ![[Pasted image 20231010062012.png]] ![[Pasted image 20231010062025.png]] ![[Pasted image 20231010062110.png]] ![[Pasted image 20231010062116.png]] ![[Pasted image 20231010062131.png]] ![[Pasted image 20231010062250.png]] ## Replit: What powers Replit AI? ![[Pasted image 20231010062943.png]] ![[Pasted image 20231010063128.png]] ![[Pasted image 20231010063149.png]] ![[Pasted image 20231010063239.png]] ![[Pasted image 20231010063321.png]] ![[Pasted image 20231010063742.png]] > 擴展受限數據的語言模型 paper https://arxiv.org/abs/2305.16264 ![[Pasted image 20231010063830.png]] ![[Pasted image 20231010064250.png]] ![[Pasted image 20231010064424.png]] ![[Pasted image 20231010064642.png]] ## Toran Bruce Richards, Inventor, AutoGPT, "The Future of Work" ![[Pasted image 20231010070004.png]] ![[Pasted image 20231010070043.png]] ## Simón Fishman, Applied AI Engineer, OpenAI & Logan Kilpatrick, Developer Relations, OpenAI, "See, Hear, Speak, Draw" ![[Pasted image 20231010070857.png]] > 本機跑一個 streamlit app 1. 先用 GPT-4V 描述一張圖片 2. 用此描丟給 DALL-E 3 產生圖片 3. GPT-4V 比較兩張圖片有什麼差異 ![[Pasted image 20231010071716.png]] ![[Pasted image 20231010071825.png]] > 給一段影片，用 whisper 轉出文字、用 GPT-4V 針對 frame 做描述，然後用 GPT-4-32K 寫篇 blog (result: https://logankilpatrick.medium.com/dont-forget-about-gpt-4-d5ab8c9493fc) ![[Pasted image 20231010072043.png]] ## Flo Crivello, CEO, Lindy, "The Age of the Agent" ![[Pasted image 20231010072350.png]] ![[Pasted image 20231010072752.png]] https://www.lindy.ai/ ![[Pasted image 20231010072940.png]] ![[Pasted image 20231010073003.png]] ## State of AI Engineering ![[Pasted image 20231010074057.png]] ![[Pasted image 20231010074120.png]] ![[Pasted image 20231010074144.png]] ![[Pasted image 20231010074229.png]] ![[Pasted image 20231010074313.png]] ![[Pasted image 20231010074359.png]] ![[Pasted image 20231010074407.png]] ![[Pasted image 20231010074425.png]] ![[Pasted image 20231010074509.png]] ![[Pasted image 20231010074519.png]] ![[Pasted image 20231010074540.png]] ![[Pasted image 20231010074616.png]] https://bit.ly/AIeng23 ## AI engineer Foundation ![[Pasted image 20231010074739.png]] ![[Pasted image 20231010075003.png]] ![[Pasted image 20231010075018.png]] https://agentprotocol.ai ![[Pasted image 20231010075058.png]] https://www.aie.foundation/ ## Harrison Chase, LangChain, "Building Context-Aware Reasoning Applications with LangChain and LangSmith" ![[Pasted image 20231010084816.png]] ![[Pasted image 20231010084939.png]] ![[Pasted image 20231010084945.png]] ![[Pasted image 20231010085046.png]] ![[Pasted image 20231010085113.png]] ![[Pasted image 20231010085211.png]] ![[Pasted image 20231010085241.png]] ![[Pasted image 20231010085247.png]] ![[Pasted image 20231010085304.png]] ![[Pasted image 20231010085430.png]] ![[Pasted image 20231010085511.png]] ![[Pasted image 20231010085524.png]] ![[Pasted image 20231010085906.png]] ![[Pasted image 20231010085916.png]] ![[Pasted image 20231010090006.png]] ![[Pasted image 20231010090134.png]] ![[Pasted image 20231010090352.png]] ## Jason Liu, Founder, Fivesixseven, "Pydantic is all you need" > 講者 Twitter: https://twitter.com/jxnlco > 講者 project: https://openai-function-call.onrender.com/ 投影片: https://tome.app/fivesixseven/pydantic-is-all-you-need-cllufwp8p08qoo75rx0omiybh ![[Pasted image 20231010090723.png]] ## Eugene Yan, Senior Applied Scientist, Amazon, "Building Blocks for LLM Systems & Products" > 講者說會釋出投影片.... 中間一大段不截圖了，講者投影片切換速度太快了 > 講者 blog: https://eugeneyan.com/ > 講者 twitter: https://twitter.com/eugeneyan > 投影片釋出: https://eugeneyan.com/speaking/ai-eng-summit/ ![[Pasted image 20231010092435.png]] ![[Pasted image 20231010092444.png]] ![[Pasted image 20231010094054.png]] ## Linus Lee, AI Lead, Notion, "The Hidden Life of Embeddings" 投影片: https://drive.google.com/file/d/1EDTQcHy336zxxfoJokQMj7lDw1R5w4oh/view ![[Pasted image 20231010094201.png]] ![[Pasted image 20231010094224.png]] ![[Pasted image 20231010094412.png]] ![[Pasted image 20231010094455.png]] ![[Pasted image 20231010094513.png]] ![[Pasted image 20231010094520.png]] Demos ![[Pasted image 20231010094555.png]] ![[Pasted image 20231010094741.png]] ![[Pasted image 20231010094912.png]] ![[Pasted image 20231010095026.png]] ![[Pasted image 20231010095338.png]] ![[Pasted image 20231010095556.png]] ![[Pasted image 20231010095615.png]] ![[Pasted image 20231010095641.png]] ![[Pasted image 20231010095658.png]] ![[Pasted image 20231010095716.png]] ![[Pasted image 20231010095736.png]] ![[Pasted image 20231010095801.png]] ![[Pasted image 20231010095850.png]] ## Brittany Walker, Principal, CRV, Chris White, CTO, Prefect & Bryan Bischof, Head of AI, Hex, "The AI Pivot: Fireside chat with Chris White of Prefect & Bryan Bischof of Hex" 這是 Panel discussion，沒投影片 ## Topic Tables ![[Pasted image 20231010103710.png]] # 10/10 ## Mario Rodriguez, VP of Product, GitHub, "Keynote: The AI Evolution" ![[Pasted image 20231011005646.png]] ![[Pasted image 20231011005947.png]] ![[Pasted image 20231011010139.png]] 接著談未來: ![[Pasted image 20231011010600.png]] ![[Pasted image 20231011010636.png]] ![[Pasted image 20231011010643.png]] ![[Pasted image 20231011010659.png]] ![[Pasted image 20231011010920.png]] ![[Pasted image 20231011011041.png]] ![[Pasted image 20231011011201.png]] ## Dedy Kredo, CPO, CodiumAI, "Move Fast, Break Nothing" ![[Pasted image 20231011011339.png]] ![[Pasted image 20231011011444.png]] Live demo: 用 Visual Studio Code 示範 CodiumAI 外掛，自動產生 tests、code 解釋跟建議等等 ## Matt Welsh, Co-Founder, Fixie.ai, "Building Reactive AI Apps" ![[Pasted image 20231011012646.png]] ![[Pasted image 20231011012656.png]] ![[Pasted image 20231011012823.png]] https://docs.ai-jsx.com/ 接著就介紹 AI.JAX is React, but for LLM Apps ## Amelia Wattenberger, Design, Adept, "Climbing the Ladder of Abstraction" How can we best combime AI + UIs? ![[Pasted image 20231011014544.png]] ![[Pasted image 20231011014643.png]] ![[Pasted image 20231011014739.png]] .... ![[Pasted image 20231011015300.png]] ![[Pasted image 20231011015328.png]] ![[Pasted image 20231011015340.png]] ... ![[Pasted image 20231011015928.png]] https://www.adept.ai/ ## Samantha Whitmore, CEO, New Computer & Jason Yuan, CTO/CDO, New Computer, "The Intelligent Interface" ![[Pasted image 20231011020643.png]] ![[Pasted image 20231011020804.png]] ![[Pasted image 20231011020836.png]] ![[Pasted image 20231011021244.png]] ![[Pasted image 20231011021440.png]] ![[Pasted image 20231011021751.png]] ## One More thing: AI Engineer World's FAIR 2024 ![[Pasted image 20231011022116.png]] https://ai.engineer/worlds-fair ## [[Shift Left-How to Become an AI Engineer from a Fullstack Background]] 在休息時間播放的，意外不錯的學習資料整理 ## Hassan El Mghari, AI Engineer, Vercel, "The Weekend AI Engineer" 一個喜歡在週末做 side projects 的工程師! ![[Pasted image 20231011030638.png]] ![[Pasted image 20231011030747.png]] >即使只是簡單的 OpenAI wrapper，各位工程師都會做~ 但是世界上還有很多人不會用 ChatGPT 的，所以流量還是有不少 ![[Pasted image 20231011032415.png]] ## Joseph Nelson, CEO, Roboflow, "120k players in a week: Lessons from the first viral CLIP app" ![[Pasted image 20231011032734.png]] https://paint.wtf/ ![[Pasted image 20231011032839.png]] ![[Pasted image 20231011033114.png]] live coding! ![[Pasted image 20231011033321.png]] ![[Pasted image 20231011033522.png]] ![[Pasted image 20231011033607.png]] ![[Pasted image 20231011033927.png]] ![[Pasted image 20231011033947.png]] ![[Pasted image 20231011034017.png]] ![[Pasted image 20231011034043.png]] ![[Pasted image 20231011034151.png]] ![[Pasted image 20231011034235.png]] ## Paul Copplestone, CEO, Supabase, "Supabase Vector: The Postgres Vector database" https://supabase.com/ open source: https://github.com/supabase/supabase 使用並支持 https://github.com/pgvector/pgvector/ 的開發 ![[Pasted image 20231011034532.png]] ![[Pasted image 20231011034910.png]] pgvector 在 v0.5 加入 HNSW 支援，快多啦! ![[Pasted image 20231011035003.png]] ![[Pasted image 20231011035056.png]] ...猛推薦 postgresql ## Visualizing Datasets: Unlocks a visual perspective on your text data before using it in downstream tasks https://github.com/YoungPhlo/visualizing-datasets-ai-engineer-fall-2023 ## Storyteller https://github.com/lgrammel/storyteller https://github.com/lgrammel/modelfusion ![[Pasted image 20231011045007.png]] ![[Pasted image 20231011045026.png]] ## Daniel Rosenwasser, PM TypeScript, Microsoft, "Pragmatic AI With TypeChat" ![[Pasted image 20231011050801.png]] ![[Pasted image 20231011050901.png]] .... ![[Pasted image 20231011051131.png]] https://microsoft.github.io/TypeChat/ ## Abi Aryan, ML Engineer & O'Reilly Author, "Domain adaptation and fine-tuning for domain-specific LLMs" ![[Pasted image 20231011052629.png]] .... ![[Pasted image 20231011053915.png]] ... ## Anton Troynikov, CTO, Chroma, "Retrieval Augmented Generation in the Wild" ![[Pasted image 20231011055107.png]] ![[Pasted image 20231011055158.png]] ![[Pasted image 20231011055204.png]] ![[Pasted image 20231011055237.png]] ![[Pasted image 20231011055306.png]] ![[Pasted image 20231011055318.png]] ![[Pasted image 20231011055328.png]] ![[Pasted image 20231011055418.png]] ![[Pasted image 20231011055546.png]] ![[Pasted image 20231011055606.png]] ![[Pasted image 20231011055742.png]] ![[Pasted image 20231011055932.png]] ![[Pasted image 20231011060131.png]] ## Jerry Liu, CEO, LlamaIndex, "Building Production-Ready RAG Applications" ![[Pasted image 20231011060343.png]] ![[Pasted image 20231011060357.png]] ![[Pasted image 20231011060424.png]] ![[Pasted image 20231011060458.png]] ![[Pasted image 20231011060704.png]] ![[Pasted image 20231011060722.png]] ![[Pasted image 20231011060834.png]] ![[Pasted image 20231011060903.png]] ![[Pasted image 20231011060938.png]] ![[Pasted image 20231011061047.png]] Optimizing RAG Systems ![[Pasted image 20231011061134.png]] ![[Pasted image 20231011061302.png]] ![[Pasted image 20231011061354.png]] ![[Pasted image 20231011061439.png]] ![[Pasted image 20231011061503.png]] ![[Pasted image 20231011061534.png]] ![[Pasted image 20231011061633.png]] ![[Pasted image 20231011061707.png]] ![[Pasted image 20231011061737.png]] ![[Pasted image 20231011061923.png]] ![[Pasted image 20231011062021.png]] ![[Pasted image 20231011062136.png]] 投影片: https://docs.google.com/presentation/d/1v7T6ejrSo87ndGeGC7tt6zeq-cftu03WWw7WL8Jskug/edit#slide=id.p ## Mithun Hunsur, Senior Engineer, Ambient, "Harnessing the Power of LLMs Locally" https://github.com/rustformers/llm ![[Pasted image 20231011071139.png]] ![[Pasted image 20231011071452.png]] ![[Pasted image 20231011072101.png]] ![[Pasted image 20231011072205.png]] ## Shreya Rajpal, Founder, Guardrails AI, Trust, but Verify ![[Pasted image 20231011072342.png]] ![[Pasted image 20231011072603.png]] ![[Pasted image 20231011072754.png]] ![[Pasted image 20231011072854.png]] ![[Pasted image 20231011072928.png]] https://github.com/ShreyaR/guardrails ![[Pasted image 20231011073056.png]] ![[Pasted image 20231011073232.png]] ![[Pasted image 20231011073410.png]] ![[Pasted image 20231011073623.png]] ![[Pasted image 20231011073721.png]] ![[Pasted image 20231011073931.png]] ![[Pasted image 20231011073956.png]] ![[Pasted image 20231011074022.png]] ![[Pasted image 20231011074042.png]] ![[Pasted image 20231011074108.png]] ![[Pasted image 20231011074117.png]] ![[Pasted image 20231011074125.png]] ![[Pasted image 20231011074130.png]] ![[Pasted image 20231011074144.png]] ![[Pasted image 20231011074152.png]] ![[Pasted image 20231011074159.png]] ![[Pasted image 20231011074217.png]] ## Simon Willison, Creator, Datasette; Co-creator, Django, "Open Questions for AI Engineering" ![[Pasted image 20231011074346.png]] ![[Pasted image 20231011074417.png]] ![[Pasted image 20231011074552.png]] 當時很難說服人這有用.... 直到.... ![[Pasted image 20231011074631.png]] ![[Pasted image 20231011074618.png]] ... ![[Pasted image 20231011074905.png]] ... ![[Pasted image 20231011075333.png]] 介紹 https://llm.datasette.io/ CLI ![[Pasted image 20231011075720.png]] ... ![[Pasted image 20231011075815.png]] ... ![[Pasted image 20231011075906.png]] ![[Pasted image 20231011075913.png]] ![[Pasted image 20231011075951.png]] ![[Pasted image 20231011080035.png]] ![[Pasted image 20231011080047.png]] ![[Pasted image 20231011080052.png]] ![[Pasted image 20231011080131.png]] ![[Pasted image 20231011080154.png]] ![[Pasted image 20231011080236.png]] 可以編譯 C ![[Pasted image 20231011080342.png]] ![[Pasted image 20231011080425.png]] ![[Pasted image 20231011080518.png]] ![[Pasted image 20231011080536.png]] ![[Pasted image 20231011080551.png]] ![[Pasted image 20231011080652.png]] ## Benjamin Dunphy, Managing Partner, Software 3.0 LLC & swyx, Latent.Space & Smol.ai, "Thank you and a special preview for 2024" ![[Pasted image 20231011080917.png]] ## 10/8 (workshop) - AI Engineering 101 - https://docs.google.com/presentation/d/19wEaJvOTR4ec40vXyWLKziMGQ861XA-QQqs8XAR4co0/edit?pli=1#slide=id.p1 * https://github.com/nheingit/AI-101 - Finetuning Lunch & Learn (1hr) - https://www.youtube.com/watch?v=3YjPGLLy7w4&t=1284s * https://gist.github.com/markhng525/d1b427e57edf10fdfd54c621e7e7df4d - [[Writing Principles for Task-Tuned Prompt Engineering]] - AI Engineering 201 (3hr) - https://docs.google.com/presentation/d/1g_7gSCJc5sK908D42bzSsYykSI6rXi1pFzyf-Ghkox8/edit#slide=id.p * Building, Evaluating, and Optimizing your RAG App for Production * https://github.com/run-llama/ai-engineer-workshop * https://github.com/run-llama/ai-engineer-workshop/blob/main/presentation.pdf * LangChain workshop * https://github.com/hwchase17/ai-engineer