* Florian 的 Advanced RAG 07: Exploring RAG for Tables (2024/3/16) * https://ai.plainenglish.io/advanced-rag-07-exploring-rag-for-tables-5c3fc0de7af6 * https://twitter.com/omarsar0/status/1755789530710339788 (2024/2/9) * [ ] paper: https://arxiv.org/abs/2402.05121 * https://twitter.com/omarsar0/status/1763187964501254492 (2024/2/29) * [ ] paper: https://arxiv.org/abs/2402.17944 * Instructor 用 GPT-4V 擷取出 markdown 格式 https://python.useinstructor.com/examples/extracting_tables/ * High-Precision RAG for Table Heavy Documents * https://medium.com/kx-systems/high-precision-rag-for-table-heavy-documents-using-langchain-unstructured-io-kdb-ai-22f7830eac9a * 表格加上文件摘要，轉成描述再 embedding * SpreadsheetLLM: Encoding Spreadsheets for Large Language Models (2024/7/12) * https://arxiv.org/abs/2407.09025v1 * Language Modeling on Tabular Data: A Survey of Foundations, Techniques and Evolution (2024/8/20) * https://arxiv.org/abs/2408.10548 ## Table Summary Retrieval * https://twitter.com/austinbv/status/1762782262096179532 (2024/2/28) * 針對表格用 LLM 做摘要 * 用摘要來做 embeddings 索引 * 但是用原本 table 文字做 prompt ## llamaindex * https://twitter.com/llama_index/status/1747289513934864493 * https://twitter.com/jerryjliu0/status/1730756401134461259 2023/12/2 三種方法比較 https://docs.llamaindex.ai/en/latest/examples/multi_modal/multi_modal_pdf_tables.html 推薦使用 Microsoft `Table Transformer` 從圖像中裁剪表格後，在用 GPT-4V 處理以獲得準確答案。相比直接用 GPT-4V 解析整頁 PDF 要好得多 * 使用 Unstructured * https://docs.llamaindex.ai/en/stable/examples/query_engine/sec_tables/tesla_10q_table.html * Advanced Tabular Data Understanding with LLMs * https://twitter.com/llama_index/status/1755034740951015668 webinar * https://twitter.com/llama_index/status/1756784462564921515 ## langchain https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_Structured_RAG.ipynb * langchain 也有一篇 https://blog.langchain.dev/benchmarking-rag-on-tables/ 結論是 Multi vector with ensemble 最好，針對 table summary * https://langchain-ai.github.io/langchain-benchmarks/notebooks/retrieval/semi_structured_benchmarking/ss_eval_multi_vector.html?ref=blog.langchain.dev * 沒有用其他套件偵測table，只是用 LLM 判斷有 table 就摘要 * 似乎沒有處理正確的 table parsing ??? * https://twitter.com/LangChainAI/status/1735350379901272289 影片跟投影片 ## self-consistency over Tabular Data https://twitter.com/jerryjliu0/status/1746937115567636675 2024/1/16 https://twitter.com/llama_index/status/1746937012798800272 Rethinking Tabular Data Understanding with Large Language Models https://arxiv.org/abs/2312.16702v1 MixSelfConsistencyQueryEngine ## Camelot: PDF Table Extraction for Humans 只限 text-based PDFs，圖片的不行喔 https://camelot-py.readthedocs.io/en/master/ 在 llamaindex finance data 範例中用到 https://twitter.com/jerryjliu0/status/1732566009574486365 https://colab.research.google.com/drive/1Y_lUUKMdC627J5EP0dK1H8NveovpYisM?usp=sharing#scrollTo=qUFUJFMjzFaR ## Table Transformer model (TATR) 將表格辨識出來成為圖片 https://huggingface.co/docs/transformers/model_doc/table-transformer https://github.com/microsoft/table-transformer https://twitter.com/llama_index/status/1730629675435835865 https://huggingface.co/spaces/nielsr/tatr-demo ## FT-ID TFT-ID (Table/Figure/Text IDentifier) is an object detection model finetuned to extract tables, figures, and text sections in academic papers https://huggingface.co/yifeihu/TFT-ID-1.0 https://x.com/hu_yifei/status/1816627442607366421 ## TAG * Text2SQL is Not Enough: Unifying AI and Databases with TAG * https://arxiv.org/abs/2408.14717 (2024/8) * https://github.com/tag-research/tag-bench * https://ai.plainenglish.io/goodbye-text2sql-why-table-augmented-generation-tag-is-the-future-of-ai-driven-data-queries-892e24e06922 * https://x.com/lianapatel_/status/1828939097487945948 (2024/8/29)