* 如何控制 LLM 的输出格式和解析其输出结果？ 2023/11/3 * https://baoyu.io/blog/prompt-engineering/how-to-parse-the-output-from-llm * https://twitter.com/llama_index/status/1709229624709025972 10/3 * https://twitter.com/llama_index/status/1720103157412647265 * 推薦了 LM Format Enforcer * https://twitter.com/jerryjliu0/status/1720127061917147376 * 這篇還列了 * Guidance * JSONFormer * Outlines * LMQL * https://minimaxir.com/2023/12/chatgpt-structured-data/ 2023/12/21 * https://gpt-index.readthedocs.io/en/stable/optimizing/advanced_retrieval/structured_outputs/structured_outputs.html 2024/5/11 * https://simmering.dev/blog/structured_output/ (2024/5/11) * openai structured outpus: https://ywctech.net/ml-ai/openai-structued-output-json-schema/ * https://towardsdatascience.com/diving-deeper-with-structured-outputs-b4a5d280c208/ (2024/9/3) * https://www.boundaryml.com/blog/structured-output-from-llms (2024/11/26) * https://www.tamingllms.com/notebooks/structured_output.html# * Aider 做的 https://aider.chat/2024/08/14/code-in-json.html * https://structuredoutputsbyexamples.com/ ## paper: Let Me Speak Freely? https://arxiv.org/abs/2408.02442 * https://x.com/_philschmid/status/1860947148348764627 (2024/11/25) * https://blog.dottxt.co/say-what-you-mean.html (2024) 質疑 * https://github.com/dottxt-ai/demos/tree/main/say-what-you-mean * https://x.com/dottxtai/status/1859666352107028835 * https://www.threads.net/@rayrayray9996/post/DDXqy-HSXQH 回擊 * https://github.com/appier-research/structure-gen/blob/main/updates.md回擊 大致結論: 結構化輸出對於 結構的分類任務 會提升表現，但是對於推理任務會下降 ## paper: Does Prompt Formatting Have Any Impact on LLM Performance? * https://arxiv.org/abs/2411.10541v1 (2024/11/15) * 不同格式可能會顯著影響模型性能 ## LM Format Enforcer https://github.com/noamgat/lm-format-enforcer ## 框架 跟 [[Prompt DSL]] 有重複 * https://github.com/jxnl/instructor * https://github.com/microsoft/TypeChat * https://github.com/guidance-ai/guidance