{"id":11900,"date":"2024-01-22T13:00:46","date_gmt":"2024-01-22T05:00:46","guid":{"rendered":"https:\/\/ihower.tw\/blog\/?p=11900"},"modified":"2025-07-19T19:34:46","modified_gmt":"2025-07-19T11:34:46","slug":"aie-gpt-4v","status":"publish","type":"post","link":"https:\/\/ihower.tw\/blog\/11900-aie-gpt-4v","title":{"rendered":"\u611b\u597d AI Engineer \u9031\u5831\u00a0\ud83d\ude80 GPT-4V \u5f71\u50cf\u89e3\u8b80 \u8a55\u6e2c\u7279\u8f2f #07"},"content":{"rendered":"\n
\"\"<\/a><\/figure>\n\n\n\n

Hello! \u4f60\u597d \ud83d\udc4b
GPT-4V \u548c Google Gemini \u6a21\u578b\u90fd\u6709\u4e86\u5f71\u50cf\u89e3\u8b80\u529f\u80fd\uff0c\u8b93\u4e4b\u524d\u9700\u8981\u7279\u5225\u7528\u6df1\u5ea6\u5b78\u7fd2\u7684\u5f71\u50cf\u8fa8\u8b58\u4efb\u52d9\uff0c\u8b8a\u6210\u4e86\u7c21\u55ae\u7684 API \u547c\u53eb\u3002\u4eca\u5929\u6211\u5011\u4f86\u6df1\u5165\u770b\u770b\u9019\u500b\u529f\u80fd\u3002<\/p>\n\n\n\n

\ud83d\udd1dxkcd: Tasks<\/a><\/h3>\n\n\n\n
\"\"\/<\/figure>\n\n\n\n

\u5148\u5206\u4eab\u4e00\u500b\u6642\u4ee3\u60c5\u61f7\uff0c\u9019\u662f xkcd \u5728 2014 \u7684\u6897\u5716\uff0c\u7576\u6642\u662f\u60f3\u8868\u9054\u5728CS\u9818\u57df\u4e2d\uff0c\u5f88\u96e3\u8ddf\u4e00\u822c\u4eba\u89e3\u91cb\u7c21\u55ae\u548c\u5e7e\u4e4e\u4e0d\u53ef\u80fd\u7684\u4efb\u52d9\u3002\u6c92\u60f3\u5230\u5341\u5e74\u5f8c\u7684\u4eca\u5929\uff0c\u9019\u500b\u5f71\u50cf\u89e3\u8b80\u7684\u4efb\u52d9\u4e5f\u5df2\u7d93\u8b8a\u6210\u7c21\u55ae\u4e86\uff0c\u4efb\u4f55 App \u53ea\u9700\u8981\u547c\u53eb GPT-4V API \u5c31\u53ef\u4ee5\u505a\u51fa\u4f86\u3002<\/p>\n\n\n\n

\ud83c\udfaf\u5f71\u50cf\u89e3\u8b80 Use Cases \u61c9\u7528\u5206\u985e<\/a><\/h3>\n\n\n\n

Greg Kamradt \u7684\u9019\u5247 tweet \u8cbc\u6587<\/a>\uff0c\u6574\u7406\u4e86\u5e38\u898b\u7684 Use Cases \u61c9\u7528\u5206\u985e\uff0c\u4ee5\u53ca\u7db2\u53cb\u7684\u793a\u7bc4\uff0c\u975e\u5e38\u8c50\u5bcc\u3002<\/p>\n\n\n\n\n\n\n\n

\ud83d\udc4d\u5fae\u8edf GPT-4V \u7814\u7a76\u5831\u544a<\/a><\/h3>\n\n\n\n

\u91dd\u5c0d GPT-4V\uff0c\u53bb\u5e74\u5fae\u8edf\u5c31\u6709\u767c\u8868\u4e00\u500b\u975e\u5e38\u8a73\u7d30\u7684\u8a55\u6e2c\u5831\u544a The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)<\/a> \u662f\u4efb\u4f55\u9700\u8981\u6df1\u5ea6\u4e86\u89e3 GPT-4V \u80fd\u529b\u7684\u5fc5\u8b80\u4e4b\u4f5c\uff0c\u9019\u4efd\u5831\u544a\u975e\u5e38\u9577166\u9801\uff0c\u53ef\u4ee5\u5148\u770b\u770b\u4ee5\u4e0b\u6458\u8981\u7cbe\u83ef:<\/p>\n\n\n\n

* \u300aGPT-4V\uff0c\u591a\u6a21\u6001\u5927\u6a21\u578b\u7684\u9ece\u660e\u300b\u8bba\u6587\u5185\u5bb9\u7cbe\u9009\u4e0e\u7ffb\u8bd1<\/a>
* \u8bd5\u8fc7GPT-4V\u540e\uff0c\u5fae\u8f6f\u5199\u4e86\u4e2a166\u9875\u7684\u6d4b\u8bc4\u62a5\u544a\uff0c\u4e1a\u5185\u4eba\u58eb\uff1a\u9ad8\u7ea7\u7528\u6237\u5fc5\u8bfb<\/a>
* \u53e6\u5916\u975e\u5e38\u63a8\u85a6 Ted Chen \u7684 GPT-4V \u5fae\u8edf\u8a55\u6e2c\u5831\u544a<\/a>\uff0cTed \u4f7f\u7528\u7e41\u9ad4\u4e2d\u6587\u518d\u8a55\u6e2c\u4e00\u6b21<\/p>\n\n\n\n

\ud83d\udc4aGPT-4V \u77e5\u8b58\u5bc6\u96c6\u7684\u8996\u89ba\u554f\u7b54\u8a55\u6e2c<\/a><\/h3>\n\n\n\n

\u9019\u7bc7\u8ad6\u6587\u7528\u77e5\u8b58\u5bc6\u96c6\u578b\u7684\u554f\u7b54\u4f86\u505a\u8a55\u6e2c\uff0c\u9019\u88e1\u6709\u8ad6\u6587\u7cbe\u9078\u7ffb\u8b6f GPT-4V \u5c60\u699c\u89c6\u89c9\u95ee\u7b54<\/a>\u3002
\u5176\u4e2d\u6709\u91dd\u5c0d Llava-v1.5-13b \u9019\u500b\u958b\u6e90\u6a21\u578b\u4e00\u8d77\u505a\u6bd4\u8f03\uff0c\u4ee5\u53ca\u900f\u904e few-shots \u7d66\u7bc4\u4f8b\u7684\u65b9\u5f0f\u4f86\u63d0\u5347\u7b54\u6848\u6e96\u78ba\u6027\u548c\u54c1\u8cea\u3002 <\/p>\n\n\n\n

\ud83d\udea7\u7528\u660e\u78ba\u554f\u984c\u548c CoT \u6280\u5de7\u89e3\u8b80\u5716\u8868<\/a><\/h3>\n\n\n\n

\u9664\u4e86\u7167\u7247\uff0cGPT-4V \u4e5f\u53ef\u4ee5\u89e3\u8b80\u9577\u689d\u5716\u3001\u6563\u9ede\u5716\u548c\u8868\u683c\u7b49\uff0c\u9019\u7bc7\u6587\u7ae0\u7528\u5177\u9ad4\u554f\u984c\u800c\u4e0d\u662f\u4e00\u822c\u6027\u554f\u984c\uff0c\u518d\u52a0\u4e0a CoT \u9010\u6b65\u63a8\u7406\u7684\u65b9\u6cd5\uff0c\u4f86\u5f97\u5230\u66f4\u597d\u7684\u7b54\u6848\u3002<\/p>\n\n\n\n

\ud83d\udc4aGemini Pro \u7684\u6bd4\u8f03\u8a55\u6e2c<\/h3>\n\n\n\n

Gemini Pro \u4e5f\u6709\u4e00\u6a23\u7684\u5f71\u50cf\u89e3\u8b80\u529f\u80fd\uff0c\u4e0a\u500b\u6708\u6709\u5169\u7bc7\u8ad6\u6587\u8a8d\u771f\u5730\u505a\u4e86\u6bd4\u8f03\u8a55\u6e2c: <\/p>\n\n\n\n

*  A Preliminary Comparison and Combination of Vision-Language Models Through Qualitative Case<\/a>
*  A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise<\/a><\/p>\n\n\n\n

\u5169\u7bc7\u8ad6\u6587\u90fd\u540c\u610f Gemini Pro \u975e\u5e38\u5177\u6709\u7af6\u722d\u529b\uff0c\u5169\u8005\u4e5f\u90fd\u6709\u4e00\u6a23\u7684\u5f31\u9ede: \u7a7a\u9593\u611f\u77e5\u80fd\u529b\u4e0d\u4f73\uff0c\u4e0d\u64c5\u9577\u7269\u9ad4\u7684\u76f8\u5c0d\u4f4d\u7f6e\uff0cOCR \u9084\u4e0d\u5920\u53ef\u9760\u3002
\u4e0d\u904e\u5169\u7bc7\u8ad6\u6587\u5c0d\u65bc\u5169\u8005\u8f38\u51fa\u98a8\u683c\u7684\u7d50\u8ad6\u537b\u5b8c\u5168\u4e0d\u540c: <\/p>\n\n\n\n

\u7b2c\u4e00\u7bc7\u8a8d\u70ba GPT-4V \u504f\u5411\u7cbe\u78ba\u548c\u7c21\u6f54\u7684\u56de\u7b54\uff0c\u800c Gemini \u5247\u64c5\u9577\u63d0\u4f9b\u8a73\u7d30\u3001\u8c50\u5bcc\u7684\u56de\u7b54\uff0c\u4e26\u9644\u5e36\u76f8\u95dc\u7684\u5716\u50cf\u548c\u93c8\u63a5\u3002
\u7b2c\u4e8c\u7bc7\u8a8d\u70ba GPT-4V \u504f\u5411\u63d0\u4f9b\u8a73\u7d30\u89e3\u91cb\u548c\u4e2d\u9593\u6b65\u9a5f\uff0c\u800c Gemini \u5247\u66f4\u559c\u6b61\u8f38\u51fa\u76f4\u63a5\u800c\u7c21\u6f54\u7684\u7b54\u6848\u3002  <\/p>\n\n\n\n

\u56e0\u70ba\u5be6\u5728\u592a\u8a6d\u7570\u4e86\uff0c\u6240\u4ee5\u6211\u81ea\u5df1\u6311\u4e86\u5e7e\u5f35\u7167\u7247\u7c21\u55ae\u5be6\u6e2c\u5f8c\uff0c\u63a8\u6e2c\u7b2c\u4e00\u7bc7\u8ad6\u6587\u61c9\u8a72\u4e0d\u662f\u76f4\u63a5\u7528 API \u8a55\u6e2c\u7684\uff0c\u800c\u662f\u7528 Bard \u7248\u672c(\u5305\u88f9\u904e\u7684\u804a\u5929app\uff0c\u6703\u6709\u52a0\u6599\u7684 system prompt)
\u4e00\u6a23\u7684\u5716\u7247\u6211\u6539\u7528 API \u5f8c\uff0cGemini Pro \u7684\u56de\u7b54\u7684\u78ba\u662f\u6bd4 GPT-4V \u66f4\u52a0\u7c21\u6f54\u3002<\/p>\n\n\n\n

\ud83d\udd25\u8996\u89ba\u4ee3\u7406\u63a7\u5236<\/h3>\n\n\n\n

\u6709\u4e00\u7a2e\u61c9\u7528\u975e\u5e38\u5438\u5f15\u773c\u7403\uff0c\u90a3\u5c31\u662f\u5229\u7528\u5f71\u50cf\u89e3\u8b80\u7684\u80fd\u529b\uff0c\u5373\u6642\u89e3\u8b80\u87a2\u5e55\u4e0a\u7684\u8cc7\u8a0a\uff0c\u7136\u5f8c\u5e6b\u4f60\u9032\u884c\u64cd\u4f5c\u63a7\u5236\u756b\u9762\u3002\u4f8b\u5982\u9019\u7bc7\u8ad6\u6587 SeeAct<\/a>\uff0c\u4ee5\u53ca\u6700\u65b0\u7684 Chrome extension:  AI-Employe<\/a><\/p>\n\n\n\n

\u4e0d\u904e\uff0c\u9664\u4e86\u5716\u7247\u89e3\u8b80\u7684\u901f\u5ea6\u9084\u4e0d\u5920\u5feb\u4e4b\u5916\uff0c\u5373\u4f7f\u662f\u6bcf\u500b\u64cd\u4f5c\u6709 75% \u7684\u6210\u529f\u7387\uff0c\u5728\u4e00\u500b\u9700\u8981\u9023\u7e8c 5 \u500b\u6b65\u9a5f\u7684\u7528\u6236\u64cd\u4f5c\u6d41\u7a0b\u4e2d\uff0c\u53ea\u6709 24% \u7684\u6a5f\u6703\u53ef\u4ee5\u4e00\u6b21\u9806\u5229\u5b8c\u6210\uff0c\u6703\u6709\u5f88\u9ad8\u7684\u6a5f\u7387\u6703\u9700\u8981\u8017\u8cbb\u6642\u9593\u548c API \u6210\u672c\u4f86\u8a66\u932f\uff0c\u5c0d\u65bc\u9019\u7a2e\u5373\u6642\u6027\u8981\u6c42\u8f03\u9ad8\u7684\u61c9\u7528\u4f86\u8aaa\u6311\u6230\u9084\u662f\u5f88\u5927\u3002<\/p>\n\n\n\n

\ud83d\udcd5Multimodality \u7406\u8ad6\u6587\u7ae0<\/a><\/h3>\n\n\n\n

\u82e5\u5c0d Multimodal \u6a21\u578b\u8a13\u7df4\u7406\u8ad6\u6709\u8208\u8da3\uff0c\u9019\u7bc7 Multimodality and Large Multimodal Models<\/a> \u662f\u975e\u5e38\u597d\u7684\u79d1\u666e\u6587\u7ae0\uff0c\u9019\u88e1\u6709\u7c21\u9ad4\u4e2d\u6587\u7ffb\u8b6f<\/a>\u3002
\u53e6\u5916\u4e5f\u63a8\u85a6 edX \u8ab2\u7a0b LLM Foundation Models from the Ground Up\uff0c\u5728 Module 4 – Beyond Text-Based LLMs: Multi-Modality \u6709\u4ecb\u7d39\u7406\u8ad6\uff0c\u9019\u662f\u6211\u7684\u7b46\u8a18<\/a>\uff0c\u6709\u5f88\u591a\u53c3\u8003\u8cc7\u6599\u3002<\/p>\n\n\n\n

—-<\/p>\n\n\n\n

<\/p>\n\n\n\n

\u6700\u5f8c\uff0c\u6211\u7684\u4e0b\u4e00\u6b21 LLM \u61c9\u7528\u958b\u767c\u5de5\u4f5c\u574a<\/a>\uff0c\u6703\u548c ALPHACamp \u5408\u4f5c\u958b\u73ed\u5728 3\/4, 3\/6, 3\/10 \u7dda\u4e0a\u76f4\u64ad\u8ab2\uff0c\u76ee\u524d\u8d85\u65e9\u9ce5\u958b\u653e\u5831\u540d\u4e2d<\/a> \u81f3 1\/26 (\u4e94) 23:59<\/p>\n\n\n\n

\u5e0c\u671b\u4f60\u6703\u559c\u6b61\u9019\u4efd\u9031\u5831(\u73fe\u5728\u8981\u8b8a\u6210\u96d9\u9031\u5831\u60f9)\uff0c\u82e5\u6709\u4efb\u4f55\u56de\u994b\u6216\u9f13\u52f5\uff0c\u6b61\u8fce\u4f86\u4fe1\u7d66\u6211\uff0c\u8b1d\u8b1d\u3002<\/p>\n\n\n\n

\u2013 ihower<\/p>\n\n\n\n

<\/a><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Hello! \u4f60\u597d \ud83d\udc4bGPT-4V \u548c Google Gemini \u6a21\u578b\u90fd\u6709\u4e86\u5f71\u50cf\u89e3\u8b80\u529f\u80fd\uff0c\u8b93\u4e4b\u524d\u9700 … <\/p>\n

\u95b1\u8b80\u5168\u6587\u3008\u611b\u597d AI Engineer \u9031\u5831\u00a0\ud83d\ude80 GPT-4V \u5f71\u50cf\u89e3\u8b80 \u8a55\u6e2c\u7279\u8f2f #07\u3009<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[91],"tags":[],"class_list":["post-11900","post","type-post","status-publish","format-standard","hentry","category-aie","entry"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p1q6tG-35W","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/posts\/11900","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/comments?post=11900"}],"version-history":[{"count":6,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/posts\/11900\/revisions"}],"predecessor-version":[{"id":13027,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/posts\/11900\/revisions\/13027"}],"wp:attachment":[{"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/media?parent=11900"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/categories?post=11900"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/tags?post=11900"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}