{"id":12509,"date":"2024-10-02T17:00:54","date_gmt":"2024-10-02T09:00:54","guid":{"rendered":"https:\/\/ihower.tw\/blog\/?p=12509"},"modified":"2025-07-04T07:44:10","modified_gmt":"2025-07-03T23:44:10","slug":"openai-devday-2024","status":"publish","type":"post","link":"https:\/\/ihower.tw\/blog\/12509-openai-devday-2024","title":{"rendered":"OpenAI DevDay 2024 \u91cd\u9ede\u6574\u7406"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"676\" data-attachment-id=\"12510\" data-permalink=\"https:\/\/ihower.tw\/blog\/12509-openai-devday-2024\/2024-10-02-03-01-40\" data-orig-file=\"https:\/\/ihower.tw\/blog\/wp-content\/uploads\/2024\/10\/2024-10-02-03.01.40.jpg\" data-orig-size=\"1088,718\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"2024-10-02 03.01.40\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/ihower.tw\/blog\/wp-content\/uploads\/2024\/10\/2024-10-02-03.01.40-300x198.jpg\" data-large-file=\"https:\/\/ihower.tw\/blog\/wp-content\/uploads\/2024\/10\/2024-10-02-03.01.40-1024x676.jpg\" src=\"https:\/\/ihower.tw\/blog\/wp-content\/uploads\/2024\/10\/2024-10-02-03.01.40-1024x676.jpg\" alt=\"\" class=\"wp-image-12510\" style=\"width:649px;height:auto\" srcset=\"https:\/\/ihower.tw\/blog\/wp-content\/uploads\/2024\/10\/2024-10-02-03.01.40-1024x676.jpg 1024w, https:\/\/ihower.tw\/blog\/wp-content\/uploads\/2024\/10\/2024-10-02-03.01.40-300x198.jpg 300w, https:\/\/ihower.tw\/blog\/wp-content\/uploads\/2024\/10\/2024-10-02-03.01.40-768x507.jpg 768w, https:\/\/ihower.tw\/blog\/wp-content\/uploads\/2024\/10\/2024-10-02-03.01.40.jpg 1088w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>\u9019\u662f OpenAI DevDay 2024 \u820a\u91d1\u5c71\u5834 10\/1 \u7684\u56db\u5927\u91cd\u9ede <img loading=\"lazy\" decoding=\"async\" height=\"16\" width=\"16\" src=\"https:\/\/static.xx.fbcdn.net\/images\/emoji.php\/v9\/tef\/2\/16\/1f525.png\" alt=\"\ud83d\udd25\"><img loading=\"lazy\" decoding=\"async\" height=\"16\" width=\"16\" src=\"https:\/\/static.xx.fbcdn.net\/images\/emoji.php\/v9\/tef\/2\/16\/1f525.png\" alt=\"\ud83d\udd25\"><img loading=\"lazy\" decoding=\"async\" height=\"16\" width=\"16\" src=\"https:\/\/static.xx.fbcdn.net\/images\/emoji.php\/v9\/tef\/2\/16\/1f525.png\" alt=\"\ud83d\udd25\"><img loading=\"lazy\" decoding=\"async\" height=\"16\" width=\"16\" src=\"https:\/\/static.xx.fbcdn.net\/images\/emoji.php\/v9\/tef\/2\/16\/1f525.png\" alt=\"\ud83d\udd25\"><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li> Realtime API \u4f60\u4e5f\u53ef\u4ee5\u505a\u51fa ChatGPT \u9032\u968e\u8a9e\u97f3\u6a21\u5f0f\u5728\u4f60\u7684 app \u4e86\uff0c\u9019\u5c07\u5e36\u4f86\u4e00\u6574\u500b\u65b0\u4e16\u4ee3\u7684\u8a9e\u97f3 app<\/li>\n\n\n\n<li>Prompt caching \u4e0d\u7528\u984d\u5916\u8a2d\u7f6e\uff0c\u547d\u4e2d\u5feb\u53d6\u5c31\u81ea\u52d5\u6709 50% \u6298\u6263<\/li>\n\n\n\n<li>Vision fine tuning \u8996\u89ba\u5716\u7247\u5fae\u8abf gpt-4o<\/li>\n\n\n\n<li>Model distillation &amp; eval tools! \u6a21\u578b\u84b8\u993e\u548c\u8a55\u4f30\u5de5\u5177<\/li>\n<\/ol>\n\n\n\n<!--more-->\n\n\n\n<p>\u4ee5\u4e0b\u7d30\u7bc0\u89e3\u8b80:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1. Realtime API \u5373\u6642\u8a9e\u97f3<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5b98\u65b9\u516c\u544a: <a href=\"https:\/\/openai.com\/index\/introducing-the-realtime-api\">openai.com\/index\/introducing-the-realtime-api<\/a><\/li>\n\n\n\n<li>\u5b98\u65b9 API \u6587\u4ef6: <a href=\"https:\/\/platform.openai.com\/docs\/guides\/realtime\">platform.openai.com\/docs\/guides\/realtime<\/a><br><\/li>\n\n\n\n<li>\u4e4b\u524d\u8981\u505a\u8a9e\u97f3\u52a9\u624b\uff0c\u9700\u8981\u7528 whisper \u5148\u505a\u8a9e\u97f3\u8fa8\u8b58\uff0c\u8f49\u6587\u5b57\u5f8c\u547c\u53eb API\uff0c\u5f97\u5230\u6587\u5b57\u5f8c\u518d\u7528 TTS API \u8f49\u6210\u8a9e\u97f3\u56de\u7d66\u7528\u6236\uff0c\u4f46\u9019\u6a23\u4e0d\u4f46\u901f\u5ea6\u8f03\u6162\uff0c\u800c\u4e14\u5931\u53bb\u60c5\u611f\u3001\u91cd\u97f3\u548c\u53e3\u97f3\u3002<\/li>\n\n\n\n<li>\u73fe\u5728\u7528 Realtime API \u53ef\u4ee5\u5b8c\u6574\u8655\u7406\u6574\u500b\u904e\u7a0b\uff0c\u76f4\u63a5\u4e32\u6d41\u97f3\u8a0a\u8f38\u5165\u548c\u8f38\u51fa\uff0c\u4f7f\u5c0d\u8a71\u9ad4\u9a57\u66f4\u52a0\u81ea\u7136\uff0c\u9084\u53ef\u4ee5\u5728\u63a5\u6536\u97f3\u8a0a\u6642\u7528 API \u63d0\u524d\u4e2d\u65b7\u3002<\/li>\n\n\n\n<li>\u5c31\u662f\u524d\u5e7e\u5929 ChatGPT \u7684 Advanced Voice mode \u7684 API \u7248\u672c\uff0c\u73fe\u5728\u5927\u5bb6\u81ea\u5df1\u7684 app \u4e5f\u53ef\u4ee5\u505a\u4e86<\/li>\n\n\n\n<li>\u5be6\u4f5c\u4e0a\uff0cRealtime API \u4f7f\u7528 WebSocket \u9023\u63a5\uff0c\u6574\u500b API \u90fd\u662f\u57fa\u65bc websoket \u4e8b\u4ef6\u4f86\u8655\u7406\uff0c\u5305\u62ec\u97f3\u8a0a\u4e5f\u662f\u7528 base64 \u7de8\u78bc\u5f8c\u585e\u9032 json \u50b3\u905e<\/li>\n\n\n\n<li>\u652f\u63f4 function calling\uff0c\u53ef\u4ee5\u7528\u4f86\u900f\u904e\u8a9e\u97f3\u4f86\u547c\u53eb tool \u5de5\u5177<\/li>\n\n\n\n<li>\u65b0\u6a21\u578b\u53eb\u505a\u662f gpt-4o-realtime-preview \u4eca\u5929\u61c9\u8a72\u5c31\u6703\u958b\u653e\u7d66\u5927\u5bb6<\/li>\n\n\n\n<li>\u9084\u6709\u53e6\u6709\u4e00\u500b gpt-4o-audio-preview \u6a21\u578b\u63a5\u4e0b\u4f86\u5e7e\u9031\u6703\u91cb\u51fa\uff0c\u6211\u4e0d\u592a\u78ba\u5b9a\u5dee\u5225\u662f\u4ec0\u9ebc\uff0c\u770b\u8d77\u4f86\u662f\u4e00\u500b\u975e realtime \u7684\u7248\u672c(\u548c Chat Completions API \u4e00\u6a23\u7684 HTTP API)\uff0c\u53ef\u4ee5\u8f38\u51fa\u548c\u8f38\u51fa \u6587\u5b57\u6216\u8a9e\u97f3<\/li>\n\n\n\n<li>\u50f9\u9322: \u6bcf\u5206\u9418\u8a9e\u97f3\u8f38\u5165\u7d04 0.06 \u7f8e\u5143\uff0c\u6bcf\u5206\u9418\u8a9e\u97f3\u8f38\u51fa\u7d04 0.24 \u7f8e\u5143\uff0c\u76ee\u524d\u9084\u662f\u4e0d\u4fbf\u5b9c\u7684<\/li>\n\n\n\n<li>\u5b98\u65b9\u7684 demo app: <a href=\"https:\/\/l.facebook.com\/l.php?u=https%3A%2F%2Fgithub.com%2Fopenai%2Fopenai-realtime-console%3Ffbclid%3DIwZXh0bgNhZW0CMTAAAR2aJLcCbBJbkUUZoArxrdsogA3gmzL_YnZLnOoxXgdWLbtQ0Pp2MrCtV2Y_aem_l4Yjl4lq2uCMRWc690P5SA&amp;h=AT1312Wb-WI1iOFHbaoUY_0-gKfIC5zdCZGGxtcYgtt1tehuqi1UrN35i5dm_x1QT6eNEl661qb4fGHyhNQBwCV_073Eu7glDvRo3-5Dekq-A5sfc6rCkDXiCpBU2yf4qvam0dc&amp;__tn__=-UK-y-R&amp;c[0]=AT0_U5s_mzM-L5e9h0s0f5j25TSPOv8gKKDP4WfFGKsU-30FFUcGx5cKKwQrmfsnnIwmj2B0RpUWZs2V9C9k3ujxX0KC0iKRi0Qlz-D-I2_CJHktz-c3oi9n-yXPusDWluvP_algWGYUJmFLuM0DmwZpZ1Yt6l8QwrHeeV_61sa5s-xVT1TC0OyyaQmpPrBsBe36u38Jairt-et7dOSUJRP2pKXXKuBQOX51tnaYKtLbCO_EWcx6P5Bi\" target=\"_blank\" rel=\"noreferrer noopener\">github.com\/openai\/openai-realtime-console<\/a> \u9019\u500b\u5168\u7528\u524d\u7aef\u505a\u7684\uff0c\u700f\u89bd\u5668\u76f4\u63a5 WebSocket \u53bb\u9023\u63a5 OpenAI \u5f8c\u7aef\uff0cAPI key \u4e5f\u5b58\u5728\u524d\u7aef\u4e0a\uff0c\u56e0\u6b64\u53ea\u662f demo \u7528\u9014\u3002\u5be6\u52d9\u4e0a production \u662f\u7528\u6236\u700f\u89bd\u5668\u9023\u63a5\u6211\u5011\u7684 app server\uff0c\u6211\u5011 server \u518d\u9023\u63a5 OpenAI \u5f8c\u7aef (\u9019 repo \u4e2d\u4e5f\u6709\u5f8c\u7aef\u65b9\u6848 Relay Server \u7684\u7bc4\u4f8b code)\u3002<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">2. Prompt caching \u5feb\u53d6<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5b98\u65b9\u516c\u544a: <a href=\"https:\/\/openai.com\/index\/api-prompt-caching\/\">openai.com\/index\/api-prompt-caching\/<\/a><\/li>\n\n\n\n<li>\u5b98\u65b9 API \u6587\u4ef6: <a href=\"https:\/\/platform.openai.com\/docs\/guides\/prompt-caching\">platform.openai.com\/docs\/guides\/prompt-caching<\/a><br><\/li>\n\n\n\n<li>\u9069\u7528 gpt-4o, gpt-4o-mini, o1-preview, o1-mini<\/li>\n\n\n\n<li>\u53ea\u6709\u5728 prompt prefix \u7b26\u5408\u6642\u624d\u6709\u4f5c\u7528\uff0c\u56e0\u6b64\u8acb\u628a\u4e0d\u6703\u8b8a\u7684 prompt \u6587\u5b57\u653e\u524d\u9762\uff0c\u6703\u8b8a\u7684\u5167\u5bb9\u653e\u5f8c\u9762<\/li>\n\n\n\n<li>\u6703\u88ab\u5feb\u53d6\u7684\u9577\u5ea6\u81f3\u5c11\u8981\u6709 1024 tokens\uff0c\u4e26\u4e14 128 tokens \u7d2f\u52a0\uff0c\u4f8b\u5982 1024, 1152, 1280&#8230;.<\/li>\n\n\n\n<li>\u56de\u50b3 response \u7684 usage \u6703\u6709 cached_tokens \u544a\u8a34\u4f60\u547d\u4e2d\u591a\u5c11 tokens\uff0c\u53ef\u4ee5\u4fbf\u5b9c\u4e00\u534a\u50f9\u9322 (\u4e0d\u904e\u76f8\u6bd4\u5176\u4ed6\u5bb6\u4f8b\u5982 Claude \u5982\u679c\u547d\u4e2d\u5feb\u53d6\u662f\u770190%\u5566)<\/li>\n\n\n\n<li>image, tool use, structured outputs json schema \u9019\u4e9b\u4e5f\u53ef\u4ee5\u7b97\u5728 \u5feb\u53d6\uff0c\u8b9a\u8b9a<\/li>\n\n\n\n<li>\u5feb\u53d6\u7684 expire \u6642\u9593\u662f 5~10 \u5206\u9418(\u5982\u679c\u6c92\u4f5c\u7528\u5c31\u6703\u88ab\u6e05\u6389)\uff0c\u975e\u9ad8\u5cf0\u6642\u9593\u6700\u591a\u5e6b\u4f60\u6d3b\u4e00\u5c0f\u6642<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">3. Vision fine tuning \u8996\u89ba\u5fae\u8abf<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5b98\u65b9\u516c\u544a: <a href=\"https:\/\/openai.com\/index\/introducing-vision-to-the-fine-tuning-api\">openai.com\/index\/introducing-vision-to-the-fine-tuning-api\/<\/a><\/li>\n\n\n\n<li>\u5b98\u65b9 API \u6587\u4ef6: <a href=\"https:\/\/platform.openai.com\/docs\/guides\/fine-tuning\/vision\">platform.openai.com\/docs\/guides\/fine-tuning\/vision<\/a><br><\/li>\n\n\n\n<li>\u53ef\u4ee5\u62ff\u4f60\u7684\u5716\u7247\u8cc7\u6599\u96c6\u53bb\u5fae\u8abf gpt-4o \u6a21\u578b<\/li>\n\n\n\n<li>\u53ef\u4ee5\u7528\u65bc\u589e\u5f37\u7684\u8996\u89ba\u641c\u7d22\u529f\u80fd\u3001\u7522\u54c1\u63a8\u85a6\u3001\u81ea\u52d5\u99d5\u99db\u8eca\u8f1b\u6216\u667a\u6167\u57ce\u5e02\u7684\u7269\u9ad4\u6aa2\u6e2c\u6539\u9032(\u4ea4\u901a\u6a19\u8a8c\u6aa2\u6e2c\u6216\u8eca\u9053\u6aa2\u6e2c)\u3001\u91ab\u7642\u5f71\u50cf\u5206\u6790\u7b49\u7b49<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">4. Model distillation \u6a21\u578b\u84b8\u993e<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5b98\u65b9\u516c\u544a: <a href=\"https:\/\/openai.com\/index\/api-model-distillation\/\">openai.com\/index\/api-model-distillation\/<\/a><\/li>\n\n\n\n<li>\u5b98\u65b9 API \u6587\u4ef6: <a href=\"https:\/\/platform.openai.com\/docs\/guides\/distillation\">platform.openai.com\/docs\/guides\/distillation<\/a><\/li>\n\n\n\n<li>\u5b98\u65b9 API \u8a55\u4f30\u6587\u4ef6: <a href=\"https:\/\/platform.openai.com\/docs\/guides\/evals\">platform.openai.com\/docs\/guides\/evals<\/a> <br><\/li>\n\n\n\n<li>\u5176\u5be6\u5c31\u662f\u505a\u5fae\u8abf\uff0c\u53ea\u662f openai \u91dd\u5c0d\u9019\u500b\u84b8\u993e\u5834\u666f\u628a\u6d41\u7a0b\u505a\u7684\u66f4\u9806<\/li>\n\n\n\n<li>\u65b0\u7684 Stored Completions \u529f\u80fd: \u5728 gpt-4o \u6216 o1 \u7684 API \u547c\u53eb\u4e2d\uff0c\u7528 store: true, metadata: {&#8220;tag&#8221;: &#8220;my-app-1&#8221;} \u53ef\u4ee5\u628a\u7d50\u679c\u5728 openai \u5f8c\u53f0\u5132\u5b58\u4e0b\u4f86\uff0c\u4e26\u642d\u914d metadata \u4e00\u8d77\uff0c\u4f8b\u5982 metadata \u5beb\u9019\u662f\u4f60\u7684\u54ea\u4e00\u500b app \u7684 prompt<\/li>\n\n\n\n<li>\u5728\u5f8c\u53f0\u900f\u904e metadata \u627e\u51fa\u9019\u4e9b input\/output pairs\uff0c\u7528\u9019\u4e9b pairs \u53bb\u5fae\u8abf gpt-4o-mini !!!<\/li>\n\n\n\n<li>\u642d\u914d\u5f8c\u53f0\u65b0\u51fa\u7684\u8a55\u4f30\u5de5\u5177\uff0c\u4f60\u5c31\u53ef\u4ee5\u628a\u4e00\u500b\u5e73\u5e38\u4f60\u7528 gpt-4o \u505a\u7279\u5b9a\u4efb\u52d9\u7684 app\uff0c\u8dd1\u4e00\u9663\u5b50\u6536\u96c6\u6578\u64da\u4e4b\u5f8c\uff0c\u5c31\u53ef\u4ee5\u63db\u6210\u5fae\u8abf\u597d\u7684 gpt-4o-mini \u4f86\u7701\u9322\u5566!!<\/li>\n<\/ul>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Whisper large-v3-turbo<\/h3>\n\n\n\n<p>\u53e6\u5916\u9084\u6709\u4e00\u500b keynote \u6c92\u8b1b\u7684\u66f4\u65b0\u662f Whisper large-v3-turbo model \u91cb\u51fa&nbsp;<a href=\"https:\/\/github.com\/openai\/whisper\/pull\/2361\/files?fbclid=IwZXh0bgNhZW0CMTAAAR2aJLcCbBJbkUUZoArxrdsogA3gmzL_YnZLnOoxXgdWLbtQ0Pp2MrCtV2Y_aem_l4Yjl4lq2uCMRWc690P5SA\" target=\"_blank\" rel=\"noreferrer noopener\">github.com\/openai\/whisper\/pull\/2361\/files<\/a><\/p>\n\n\n\n<p>\u9019\u662f large-v3 \u7684\u84b8\u993e\u7248\u672c\uff0c\u901f\u5ea6\u66f4\u5feb\uff0c\u72a7\u7272\u4e00\u9ede\u9ede\u6e96\u78ba\u6027\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u53c8\u4ed4\u7d30\u770b\u4e86 Realtime API \u6587\u4ef6<\/h3>\n\n\n\n<p>\u9664\u4e86 OpenAI \u7684\u5b98\u65b9\u6587\u4ef6\uff0c\u4e5f\u63a8\u85a6 Azure <a href=\"https:\/\/github.com\/azure-samples\/aoai-realtime-audio-sdk?fbclid=IwZXh0bgNhZW0CMTAAAR3uN4T4fZOtVthzCQDtRQIi5hRL9YpZidmH8zg6oSsDRqENA94FTyGwZjU_aem_LxofVGIVWQSTkLHHSgbnTQ\" target=\"_blank\" rel=\"noreferrer noopener\">github.com\/azure-samples\/aoai-realtime-audio-sdk<\/a>&nbsp;README \u5beb\u7684\u4e0d\u932f\u53ef\u4ee5\u4e00\u770b<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\u8f38\u5165\u53ef\u4ee5\u662f text \u6216 audio\uff0c\u4f60\u53ef\u4ee5\u53ea\u6253\u5b57\u8f38\u5165\u4e0d\u8b1b\u8a71\n<ul class=\"wp-block-list\">\n<li>demo: <a href=\"https:\/\/www.facebook.com\/ihower\/videos\/1054935976267895\">www.facebook.com\/ihower\/videos\/1054935976267895<\/a><\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>\u8f38\u51fa\u53ef\u4ee5\u662f audio \u6216 (audio+text) (\u4e5f\u5c31\u662f\u540c\u6642\u4e32\u6d41\u8072\u97f3\u8ddf\u6587\u5b57\u90fd\u7d66\u4f60)\uff0c\u4f46\u662f\u4e0d\u80fd\u53ea\u8f38\u51fa text\uff0c\u9019\u500b\u53c3\u6578\u53eb\u505a modalities\uff0c\u9810\u8a2d\u662f [&#8220;text&#8221;, &#8220;audio&#8221;]<\/li>\n\n\n\n<li>\u8f38\u5165\u662f audio \u6642\uff0c\u8f38\u51fa\u53ef\u4ee5\u591a transcription \u6587\u5b57 (\u4e5f\u5c31\u662f\u4f60\u8b1b\u8a71\u8072\u97f3\u7684\u9010\u5b57\u7a3f)\uff0c\u9019\u500b\u53c3\u6578\u662f input_audio_transcription \u53ef\u4ee5\u958b\u95dc<\/li>\n\n\n\n<li>\u53ef\u4ee5\u8a2d\u5b9a\u662f\u5426\u6253\u958b VAD (voice activity detection) \u6a21\u5f0f\uff0c\u4e5f\u5c31\u662f\u59cb\u7d42\u5728\u4e32\u6d41\u5c0d\u8a71\u4e26\u8b93 AI \u5224\u65b7\u81ea\u52d5\u56de\u8a71\u3002\u53ef\u4ee5\u8a2d\u5b9a activation threshold, prefix_padding_ms, silence_duration_ms \u4f86\u8abf\u6574\u654f\u611f\u5ea6\u3002<br>\u9019\u4e5f\u53ef\u4ee5\u95dc\u9589\uff0c\u7528\u5728\u4f60\u81ea\u5df1\u505a\u500b\u901a\u8a71\u6309\u9215\uff0c\u624b\u52d5\u63a7\u5236\u4f60\u8b1b\u8a71\u7684\u958b\u59cb\u8ddf\u7d50\u675f\u3002<\/li>\n\n\n\n<li>\u540c\u4e00\u500b websocket session \u9023\u7dda AI\u6703\u8a18\u5f97\u4f60\u8b1b\u904e\u4ec0\u9ebc\u3002\u4f46\u662f\u5982\u679c user \u4e2d\u65b7\u91cd\u9023\uff0cAI \u4e0d\u6703\u8a18\u5f97\u4e4b\u524d\u8b1b\u904e\u5565\uff0c\u4f60\u53ef\u4ee5\u628a\u5c0d\u8a71\u7d00\u9304\u900f\u904e websocket \u4e8b\u4ef6\u5148\u50b3\u7d66 AI\uff0c\u4f46\u662f\u9019\u88e1\u4e0d\u652f\u63f4\u50b3\u8a9e\u97f3! \u56e0\u6b64\u4f60\u5f97\u81ea\u5df1\u5b58\u4e0b\u6240\u6709\u5c0d\u8a71\u6587\u5b57(\u96e3\u602a\u9019\u500b API \u90fd\u6709\u529f\u80fd\u53ef\u4ee5\u62ff\u5230 text)<\/li>\n\n\n\n<li>\u60f3\u624b\u52d5\u63d0\u524d\u4e2d\u65b7AI\u8f38\u51fa\u7684\u8a71\uff0c\u9664\u4e86\u547c\u53eb response.cancel \u4e8b\u4ef6\u4e4b\u5916\uff0c\u9084\u6709\u4e00\u500b conversation.item.truncate \u4e8b\u4ef6\u53ef\u4ee5\u547c\u53eb\uff0c\u7528\u9014\u662f\u53ef\u4ee5\u522a\u6389 AI \u9084\u6c92\u8b1b\u5230\u7684\u8a71 (\u56e0\u70ba AI \u5f88\u53ef\u80fd\u5df2\u7d93\u751f\u6210\u597d\u5167\u5bb9\u585e\u9032\u5c0d\u8a71 \u6b77\u53f2\u7d00\u9304\u4e86\uff0c\u4f46\u662f\u56e0\u70ba\u8a9e\u97f3\u8b1b\u4e0d\u5920\u5feb\uff0c\u7528\u6236\u5176\u5be6\u9084\u6c92\u807d\u5230)<\/li>\n\n\n\n<li>\u5c0f\u7d50: \u8f38\u51fa\u7684\u6587\u5b57\u6709\u5169\u7a2e: \n<ul class=\"wp-block-list\">\n<li> \u4e00\u7a2e\u662f\u900f\u904e response.content_part.* \u4e8b\u4ef6\u4e32\u6d41\u56de\u4f86\u7684\uff0c\u9019\u7a2e\u662f gpt-4o-realtime-preview \u6a21\u578b\u7684\u8f38\u51fa<\/li>\n\n\n\n<li> \u4e00\u7a2e\u662f\u900f\u904e conversation.item.input_audio_transcription.completed \u4e8b\u4ef6\u56de\u4f86\u7684\uff0c\u9019\u7a2e\u662f\u7528 whisper \u6a21\u578b\u628a\u4f60\u7684\u8a9e\u97f3\u8f38\u5165\u8f49\u6210\u9010\u5b57\u7a3f\u3002<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<p>(2024\/10\/4) OpenAI \u7684 Realtime API \u7d42\u65bc\u4e0a\u7dda\u5566\uff0cpaid tiers \u958b\u767c\u8005\u61c9\u8a72\u90fd\u6709\u6b0a\u9650\u4e86\u3002<\/p>\n\n\n\n<p>\u5f8c\u53f0\u7684 Playground \u5c31\u6709 Realtime \u529f\u80fd\u53ef\u4ee5\u73a9\uff0c\u505a\u7684\u883b\u4e0d\u932f\u7684\uff0c\u53ef\u4ee5\u8abf\u6574\u7684\u53c3\u6578\u90fd\u6709\u53ef\u4ee5\u64cd\u4f5c! \u5305\u62ec\u6539 System Instruction, \u6539 Voice, \u95dc\u9589 VAD \u6a21\u5f0f\u6539\u6210\u624b\u52d5\u63a7\u5236, \u53ef\u4ee5\u6587\u5b57\u8f38\u5165, Function Calling \u6e2c\u8a66\u7b49\u7b49\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u9019\u662f OpenAI DevDay 2024 \u820a\u91d1\u5c71\u5834 10\/1 \u7684\u56db\u5927\u91cd\u9ede<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[80],"tags":[],"class_list":["post-12509","post","type-post","status-publish","format-standard","hentry","category-llm","entry"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p1q6tG-3fL","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/posts\/12509","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/comments?post=12509"}],"version-history":[{"count":8,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/posts\/12509\/revisions"}],"predecessor-version":[{"id":12523,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/posts\/12509\/revisions\/12523"}],"wp:attachment":[{"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/media?parent=12509"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/categories?post=12509"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ihower.tw\/blog\/wp-json\/wp\/v2\/tags?post=12509"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}