{"id":35490,"date":"2024-12-25T23:00:02","date_gmt":"2024-12-25T15:00:02","guid":{"rendered":"https:\/\/17aitech.com\/?p=35490"},"modified":"2024-12-25T23:00:02","modified_gmt":"2024-12-25T15:00:02","slug":"%e8%bf%99%e5%b0%b1%e7%bf%bb%e8%bd%a6%e4%ba%86%ef%bc%9freflection-70b%e9%81%ad%e8%b4%a8%e7%96%91%e5%9f%ba%e6%a8%a1%e4%b8%ballama-3%ef%bc%8c%e4%bd%9c%e8%80%85%ef%bc%9a%e9%87%8d%e6%96%b0%e8%ae%ad","status":"publish","type":"post","link":"https:\/\/17aitech.com\/?p=35490","title":{"rendered":"\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3"},"content":{"rendered":"<p>\u6587\u7ae0\u6765\u6e90\u4e8e\u4e92\u8054\u7f51:<a href=\"https:\/\/www.jiqizhixin.com\/articles\/2024-09-08-8\" target=\"_blank\">\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3<\/a><\/p>\n<p>\u6700\u8fd1\uff0c\u5f00\u6e90\u5927\u6a21\u578b\u793e\u533a\u518d\u6b21\u300c\u70ed\u95f9\u300d\u4e86\u8d77\u6765\uff0c\u4e3b\u89d2\u662f AI \u5199\u4f5c\u521d\u521b\u516c\u53f8 HyperWrite \u5f00\u53d1\u7684\u65b0\u6a21\u578b <a data-itemshowtype=\"0\" data-linktype=\"2\" href=\"http:\/\/mp.weixin.qq.com\/s?__biz=MzA3MzI4MjgzMw==&amp;mid=2650933233&amp;idx=1&amp;sn=5bc95222bcb2ab21995db9a52e9acffb&amp;chksm=84e7c38fb3904a99dcbca34c0fcf4979f247f308535eeb08abca9208750f461209baaf6e02d4&amp;scene=21#wechat_redirect\" target=\"_blank\">Reflection 70B<\/a>\u3002<\/p>\n<p>\u5b83\u7684\u5e95\u5c42\u6a21\u578b\u5efa\u7acb\u5728 Meta Llama 3.1 70B Instruct \u4e0a\uff0c\u5e76\u4f7f\u7528\u539f\u59cb\u7684 Llama chat \u683c\u5f0f\uff0c\u786e\u4fdd\u4e86\u4e0e\u73b0\u6709\u5de5\u5177\u548c pipeline \u7684\u517c\u5bb9\u6027\u3002<\/p>\n<p>\u8fd9\u4e2a\u6a21\u578b\u6a2a\u626b\u4e86 MMLU\u3001MATH\u3001IFEval\u3001GSM8K\uff0c\u5728\u6bcf\u9879\u57fa\u51c6\u6d4b\u8bd5\u4e0a\u90fd\u8d85\u8fc7\u4e86 GPT-4o\uff0c\u8fd8\u51fb\u8d25\u4e86 405B \u7684 Llama 3.1\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-d6127db92238e6e7d4a0a7b879f2cbd1.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-d6127db92238e6e7d4a0a7b879f2cbd1.png\"><\/a><\/p>\n<p>\u51ed\u501f\u5982\u6b64\u60ca\u8273\u7684\u6548\u679c\uff0cReflection 70B \u88ab\u51a0\u4ee5\u5f00\u6e90\u5927\u6a21\u578b\u65b0\u738b\u3002\u8be5\u6a21\u578b\u66f4\u662f\u7531\u4e24\u4f4d\u5f00\u53d1\u8005\uff08HyperWrite CEO Matt Shumer \u548c Glaive AI \u521b\u59cb\u4eba Sahil Chaudhary\uff09\u82b1\u4e86 3 \u5468\u5b8c\u6210\uff0c\u6548\u7387\u53ef\u8c13\u60ca\u4eba\u3002<\/p>\n<p>Reflection 70B \u80fd\u4e0d\u80fd\u7ecf\u53d7\u4f4f\u793e\u533a\u7684\u8003\u9a8c\u5462\uff1f\u4eca\u5929 AI \u6a21\u578b\u72ec\u7acb\u5206\u6790\u673a\u6784 Artificial Analysis \u8fdb\u884c\u4e86\u72ec\u7acb\u8bc4\u4f30\u6d4b\u8bd5\uff0c\u7ed3\u679c\u6709\u70b9\u51fa\u4e4e\u610f\u6599\u3002<\/p>\n<p>\u8be5\u673a\u6784\u8868\u793a\uff0c<strong>Reflection Llama 3.1 70B \u7684 MMLU \u5f97\u5206\u4ec5\u4e0e Llama 3 70B \u76f8\u540c\uff0c\u5e76\u4e14\u660e\u663e\u4f4e\u4e8e Llama 3.1 70B<\/strong>\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-429a6e145ba62e9e1613a77ef0f49d25.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-429a6e145ba62e9e1613a77ef0f49d25.png\"><\/a><\/p>\n<p><em><sup>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u56fe\u6e90\uff1ahttps:\/\/x.com\/ArtificialAnlys\/status\/1832505338991395131<\/sup><\/em><\/p>\n<p>\u8fd8\u6709\u79d1\u5b66\u63a8\u7406\u4e0e\u77e5\u8bc6\uff08GPQA\uff09\u548c\u5b9a\u91cf\u63a8\u7406\uff08MATH\uff09\u57fa\u51c6\u6d4b\u8bd5\u7684\u7ed3\u679c\uff0c\u540c\u6837\u4e0d\u5982 Llama 3.1 70B\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-be6b04f22bac7284def80cc1388e7288.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-be6b04f22bac7284def80cc1388e7288.png\"><\/a><\/p>\n<p><em><sup>\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u56fe\u6e90\uff1ahttps:\/\/x.com\/ArtificialAnlys\/status\/1832457791010959539<\/sup><\/em><\/p>\n<p>\u6b64\u5916\uff0cReddit \u4e0a LocalLLaMA \u793e\u533a\u7684\u4e00\u4e2a\u5e16\u5b50\u6bd4\u8f83\u4e86 Reflection\u00a070B \u4e0eLlama 3.1\u3001Llama 3 \u6743\u91cd\u7684\u5dee\u5f02\uff0c\u7ed3\u679c\u663e\u793a\uff0c<strong>Reflection \u6a21\u578b\u4f3c\u4e4e\u662f\u4f7f\u7528\u4e86\u7ecf\u8fc7\u00a0LoRA\u00a0\u8c03\u6574\u7684 Llama 3 \u800c\u4e0d\u662f Llama 3.1<\/strong>\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-43d2e1ff46ff56a8032065260ab25e01.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-43d2e1ff46ff56a8032065260ab25e01.png\"><\/a><\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-52665935ad3314ac450b4c5d8a44ebb3.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-52665935ad3314ac450b4c5d8a44ebb3.png\"><\/a><\/p>\n<p>\u8d34\u4e3b\u8fd8\u63d0\u4f9b\u4e86\u4ee5\u4e0a\u6a21\u578b\u6743\u91cd\u6bd4\u8f83\u7ed3\u679c\u7684\u4ee3\u7801\u6765\u6e90\u3002<\/p>\n<section>\n<pre data-lang=\"python\"><code>from transformers <\/code>\r\n<code>import AutoModelForCausalLM, AutoTokenizer<\/code>\r\n<code>import torch<\/code>\r\n<code>import matplotlib.pyplot as plt<\/code>\r\n<code>import seaborn as sns<\/code>\r\n<code>\r\n<\/code><code>base_model_name = \"meta-llama\/Meta-Llama-3-70B-Instruct\"<\/code>\r\n<code>chat_model_name = \"mattshumer\/Reflection-Llama-3.1-70B\"<\/code>\r\n<code>base_model = AutoModelForCausalLM.from_pretrained(base_model_name, torch_dtype=torch.bfloat16)<\/code>\r\n<code>chat_model = AutoModelForCausalLM.from_pretrained(chat_model_name, torch_dtype=torch.bfloat16)<\/code>\r\n<code>\r\n<\/code><code>def calculate_weight_diff(base_weight, chat_weight):<\/code>\r\n<code>    return torch.abs(base_weight - chat_weight).mean().item()<\/code>\r\n<code>\r\n<\/code><code>def calculate_layer_diffs(base_model, chat_model):<\/code>\r\n<code>    layer_diffs = []<\/code>\r\n<code>    for base_layer, chat_layer in zip(base_model.model.layers, chat_model.model.layers):<\/code>\r\n<code>        layer_diff = {<\/code>\r\n<code>            'input_layernorm': calculate_weight_diff(base_layer.input_layernorm.weight, chat_layer.input_layernorm.weight),<\/code>\r\n<code>            # 'mlp_down_proj': calculate_weight_diff(base_layer.mlp.down_proj.weight, chat_layer.mlp.down_proj.weight),<\/code>\r\n<code>            # 'mlp_gate_proj': calculate_weight_diff(base_layer.mlp.gate_proj.weight, chat_layer.mlp.gate_proj.weight),<\/code>\r\n<code>            # 'mlp_up_proj': calculate_weight_diff(base_layer.mlp.up_proj.weight, chat_layer.mlp.up_proj.weight),<\/code>\r\n<code>            'post_attention_layernorm': calculate_weight_diff(base_layer.post_attention_layernorm.weight, chat_layer.post_attention_layernorm.weight),<\/code>\r\n<code>            'self_attn_q_proj': calculate_weight_diff(base_layer.self_attn.q_proj.weight, chat_layer.self_attn.q_proj.weight),<\/code>\r\n<code>            'self_attn_k_proj': calculate_weight_diff(base_layer.self_attn.k_proj.weight, chat_layer.self_attn.k_proj.weight),<\/code>\r\n<code>            'self_attn_v_proj': calculate_weight_diff(base_layer.self_attn.v_proj.weight, chat_layer.self_attn.v_proj.weight),<\/code>\r\n<code>            'self_attn_o_proj': calculate_weight_diff(base_layer.self_attn.o_proj.weight, chat_layer.self_attn.o_proj.weight)<\/code>\r\n<code>        }<\/code>\r\n<code>        layer_diffs.append(layer_diff)<\/code>\r\n<code>    return layer_diffs<\/code>\r\n<code>\r\n<\/code><code>def visualize_layer_diffs(layer_diffs):<\/code>\r\n<code>    num_layers = len(layer_diffs)<\/code>\r\n<code>    num_components = len(layer_diffs[0])<\/code>\r\n<code>\r\n<\/code><code>    fig, axs = plt.subplots(1, num_components, figsize=(24, 8))<\/code>\r\n<code>    fig.suptitle(f\"{base_model_name} &lt;&gt; {chat_model_name}\", fontsize=16)<\/code>\r\n<code>\r\n<\/code><code>    for i, component in enumerate(layer_diffs[0].keys()):<\/code>\r\n<code>        component_diffs = [[layer_diff[component]] for layer_diff in layer_diffs]<\/code>\r\n<code>        sns.heatmap(component_diffs, annot=True, fmt=\".6f\", cmap=\"YlGnBu\", ax=axs[i], cbar_kws={\"shrink\": 0.8})<\/code>\r\n<code>        axs[i].set_title(component)<\/code>\r\n<code>        axs[i].set_xlabel(\"Layer\")<\/code>\r\n<code>        axs[i].set_ylabel(\"Difference\")<\/code>\r\n<code>        axs[i].set_xticks([])<\/code>\r\n<code>        axs[i].set_yticks(range(num_layers))<\/code>\r\n<code>        axs[i].set_yticklabels(range(num_layers))<\/code>\r\n<code>        axs[i].invert_yaxis()<\/code>\r\n<code>\r\n<\/code><code>    plt.tight_layout()<\/code>\r\n<code>    plt.show()<\/code>\r\n<code>\r\n<\/code><code>layer_diffs = calculate_layer_diffs(base_model, chat_model)<\/code>\r\n<code>\r\n<\/code><code>visualize_layer_diffs(layer_diffs)<\/code><\/pre>\n<\/section>\n<p>\u8fd8\u6709\u4eba\u8d34\u51fa\u4e86 Matt Shumer \u5728 Hugging Face \u5bf9 Reflection 70B \u914d\u7f6e\u6587\u4ef6\u540d\u79f0\u7684\u66f4\u6539\uff0c\u53ef\u4ee5\u770b\u5230\u4ece Llama 3 70B Instruct \u5230 Llama 3.1 70B Instruct \u7684\u53d8\u5316\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-fe03effed5048108dbe023f5c1a69dd4.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-fe03effed5048108dbe023f5c1a69dd4.png\"><\/a><\/p>\n<p>\u8fd9\u6837\u7684\u4e8b\u5b9e\u6446\u5728\u773c\u524d\uff0c\u4f3c\u4e4e\u8ba9\u4eba\u4e0d\u5f97\u4e0d\u4fe1\u3002\u5404\u8def\u7f51\u53cb\u4e5f\u5f00\u59cb\u53d1\u58f0\u9644\u548c\uff0c\u6709\u4eba\u8868\u793a\u81ea\u5df1\u4ece\u4e00\u5f00\u59cb\u5c31\u6000\u7591\u5b83\u662f Llama 3\uff0c\u5f53\u7528\u5fb7\u8bed\u95ee\u6a21\u578b\u4e00\u4e9b\u4e8b\u60c5\u65f6\uff0c\u5b83\u5374\u7528\u82f1\u8bed\u56de\u7b54\u3002\u8fd9\u79cd\u884c\u4e3a\u5bf9\u4e8e Llama 3 \u975e\u5e38\u5e38\u89c1\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-2d80b1f120b42adc945a5bcecf622b63.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-2d80b1f120b42adc945a5bcecf622b63.png\"><\/a><\/p>\n<p>\u8fd8\u6709\u4eba\u5947\u602a\u4e3a\u4ec0\u4e48 Reflection 70B \u6a21\u578b\u4e00\u5f00\u59cb\u5c31\u5f97\u5230\u4e86\u5982\u6b64\u591a\u7684\u7092\u4f5c\u548c\u5173\u6ce8\uff0c\u6bd5\u7adf\u7b2c\u4e00\u4e2a\u8c08\u8bba\u5b83\u662f\u300c\u9876\u7ea7\u5f00\u6e90\u6a21\u578b\u300d\u7684\u4eba\u662f\u5f00\u53d1\u8005\u672c\u4eba\uff08Matt\uff09\u3002\u800c\u4e14\u66f4\u786e\u5207\u5730\u8bf4\uff0c\u6a21\u578b\u662f\u5fae\u8c03\u7684\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-594ce93939563ff5fcd9aa1a2fd36a9e.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-594ce93939563ff5fcd9aa1a2fd36a9e.png\"><\/a><\/p>\n<p>\u66f4\u6709\u4eba\u5f00\u59cb\u8d28\u7591\u5f00\u53d1\u8005\uff08Matt\uff09\uff0c\u8ba4\u4e3a\u4ed6\u53ea\u662f\u8fd9\u5bb6\u516c\u53f8\uff08GlaiveAI\uff09\u7684\u5229\u76ca\u76f8\u5173\u8005\uff0c\u8bd5\u56fe\u901a\u8fc7\u7092\u4f5c\u6765\u589e\u52a0\u4ef7\u503c\uff0c\u5b9e\u9645\u4e0a\u5374\u5bf9\u8fd9\u9879\u6280\u672f\u4e00\u65e0\u6240\u77e5\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-116d44d61b43c34a470c5fb1dad2ba9c.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-116d44d61b43c34a470c5fb1dad2ba9c.png\"><\/a><\/p>\n<p>\u5728\u88ab\u8d28\u7591 Reflection 70B \u7684\u57fa\u7840\u6a21\u578b\u53ef\u80fd\u662f Llama 3 \u800c\u975e Llama 3.1 70B \u65f6\uff0cMatt Shumer \u5750\u4e0d\u4f4f\u4e86\uff0c\u73b0\u8eab\u8fdb\u884c\u4e86\u6f84\u6e05\uff0c\u5e76\u8868\u793a\u662f <strong>Hugging Face \u6743\u91cd\u51fa\u73b0\u4e86\u95ee\u9898<\/strong>\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-acb70ce9b58835da2a1ba21fd7968392.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-acb70ce9b58835da2a1ba21fd7968392.png\"><\/a><\/p>\n<p>\u5c31\u5728\u51e0\u4e2a\u5c0f\u65f6\u524d\uff0cMatt Shumer \u79f0\u5df2\u7ecf\u91cd\u65b0\u4e0a\u4f20\u4e86\u6743\u91cd\uff0c\u4f46\u4ecd\u7136\u5b58\u5728\u95ee\u9898\u3002\u540c\u65f6\u4ed6\u4eec<strong>\u5f00\u59cb\u91cd\u65b0\u8bad\u7ec3\u6a21\u578b\u5e76\u4e0a\u4f20\uff0c\u4ece\u800c\u6d88\u9664\u4efb\u4f55\u53ef\u80fd\u51fa\u73b0\u7684\u95ee\u9898\uff0c\u5e94\u8be5\u5f88\u5feb\u5c31\u4f1a\u5b8c\u6210<\/strong>\u3002<\/p>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-925d7ae7b5eb117f9414ba25df127f29.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-925d7ae7b5eb117f9414ba25df127f29.png\"><\/a><\/p>\n<section>\u5f53\u88ab\u95ee\u5230\u4e3a\u4f55\u9700\u8981\u91cd\u65b0\u8bad\u7ec3\u65f6\uff0cMatt Shumer \u8868\u793a\u672c\u4e0d\u9700\u8981\u8fd9\u6837\u505a\uff0c\u4f46\u5df2\u7ecf\u5c1d\u8bd5\u4e86\u6240\u6709\u65b9\u6cd5\u3002\u65e0\u8bba\u505a\u4ec0\u4e48\uff0cHugging Face \u4e0a Reflection 70B \u6a21\u578b\u90fd\u4f1a\u51fa\u73b0\u95ee\u9898\uff0c\u5bfc\u81f4\u79bb\u9884\u671f\u4e2d\u7684\u6027\u80fd\u5dee\u5f97\u8fdc\u3002<\/section>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-e6741d6dd6ad94b45d12e7b6039acbba.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-e6741d6dd6ad94b45d12e7b6039acbba.png\"><\/a><\/p>\n<section>\u5f53\u7136 Matt Shumer \u8fd8\u9762\u4e34\u66f4\u591a\u8d28\u7591\uff0c\u6bd4\u5982\u5bf9 GlaiveAI \u7684\u6295\u8d44\u60c5\u51b5\u3001\u4e3a\u4ec0\u4e48 Hugging Face \u4e0a\u7684\u57fa\u7840\u6a21\u578b\u4e3a Llama 3 \u800c\u4e0d\u662f Llama 3.1 \u4ee5\u53ca\u57fa\u51c6\u4e2d\u6709\u5173 LORAing \u7684\u95ee\u9898\u3002<\/section>\n<p><a href=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-51da78e26f0f283170c5afde5c3bfda4.png\" data-fancybox=\"images\" data-fancybox=\"gallery\"><img decoding=\"async\" src=\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-51da78e26f0f283170c5afde5c3bfda4.png\"><\/a><\/p>\n<section>Matt Shumer \u4e00\u4e00\u8fdb\u884c\u4e86\u89e3\u91ca\u3002\uff08\u4ee5\u4e0b\u6807\u5f15\u7528\uff09<\/section>\n<section><\/section>\n<blockquote data-author-name=\"\" data-content-utf8-length=\"233\" data-source-title=\"\" data-type=\"2\" data-url=\"\">\n<section>\n<section>\n<section>1. \u6211\u662f\u4e00\u4e2a\u8d85\u7ea7\u5c0f\u7684\u6295\u8d44\u8005\uff081000 \u7f8e\u5143\uff09\uff0c\u53ea\u662f\u4e00\u6b21\u652f\u6301\u6027\u7684\u6295\u8d44\uff0c\u56e0\u4e3a\u6211\u8ba4\u4e3a Sahil Chaudhary \u5f88\u68d2\u3002<\/section>\n<section>2. \u81f3\u4e8e\u4e3a\u4ec0\u4e48\u57fa\u7840\u6a21\u578b\u662f Llama 3\uff0c\u6211\u4eec\u4e0d\u77e5\u9053\u3002\u8fd9\u5c31\u662f\u4e3a\u4ec0\u4e48\u6211\u4eec\u4ece\u5934\u5f00\u59cb\u518d\u8bad\u7ec3\uff0c\u5e94\u8be5\u5f88\u5feb\u5b8c\u6210\u3002\u00a0<\/section>\n<section>3. \u90a3\u4e9b\u5c1d\u8bd5\u4e86 Playground \u5e76\u62e5\u6709\u65e9\u671f\u8bbf\u95ee\u6743\u9650\u7684\u7528\u6237\u83b7\u5f97\u4e86\u4e0e\u6258\u7ba1 API \u622a\u7136\u4e0d\u540c\u7684\u4f53\u9a8c\uff0c\u6211\u4eec\u9700\u8981\u5f04\u6e05\u695a\u8fd9\u4e00\u70b9\u3002\u00a0<\/section>\n<section>4. \u4e0d\u786e\u5b9a\u4ec0\u4e48\u662f LORAing\uff0c\u4f46\u6211\u4eec\u68c0\u67e5\u4e86\u6c61\u67d3\uff0c\u5c06\u5728\u4e0b\u5468\u4e0e 405B\uff08\u6216\u66f4\u65e9\uff09\u4e00\u8d77\u53d1\u5e03\u6570\u636e\u96c6\uff0c\u5230\u65f6\u5019\u53ef\u4ee5\u67e5\u770b\u3002<\/section>\n<\/section>\n<\/section>\n<\/blockquote>\n<section><\/section>\n<section>\u81f3\u4e8e\u91cd\u65b0\u8bad\u7ec3\u540e\u7684 Reflection 70B \u8868\u73b0\u5982\u4f55\uff1f\u6211\u4eec\u62ed\u76ee\u4ee5\u5f85\u3002<\/section>\n<section><\/section>\n<section><em><sup>\u53c2\u8003\u94fe\u63a5\uff1a<\/sup><\/em><\/section>\n<section><em><sup>https:\/\/www.reddit.com\/r\/LocalLLaMA\/comments\/1fb6jdy\/reflectionllama3170b_is_actually_llama3\/<\/sup><\/em><\/section>\n<p>\u6587\u7ae0\u6765\u6e90\u4e8e\u4e92\u8054\u7f51:<a href=\"https:\/\/www.jiqizhixin.com\/articles\/2024-09-08-8\" target=\"_blank\">\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u6587\u7ae0\u6765\u6e90\u4e8e\u4e92\u8054\u7f51:\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fRefle [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[27],"tags":[41,68],"class_list":["post-35490","post","type-post","status-publish","format-standard","hentry","category-news","tag-matplotlib","tag-68"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3 - \u4e00\u8d77AI\u6280\u672f<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/17aitech.com\/?p=35490\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/17aitech.com\/?p=35490\",\"url\":\"https:\/\/17aitech.com\/?p=35490\",\"name\":\"\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3 - \u4e00\u8d77AI\u6280\u672f\",\"isPartOf\":{\"@id\":\"https:\/\/17aitech.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/17aitech.com\/?p=35490#primaryimage\"},\"image\":{\"@id\":\"https:\/\/17aitech.com\/?p=35490#primaryimage\"},\"thumbnailUrl\":\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-d6127db92238e6e7d4a0a7b879f2cbd1.png\",\"datePublished\":\"2024-12-25T15:00:02+00:00\",\"author\":{\"@id\":\"https:\/\/17aitech.com\/#\/schema\/person\/60225458499e817ae0af73e67e440b9d\"},\"breadcrumb\":{\"@id\":\"https:\/\/17aitech.com\/?p=35490#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/17aitech.com\/?p=35490\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/17aitech.com\/?p=35490#primaryimage\",\"url\":\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-d6127db92238e6e7d4a0a7b879f2cbd1.png\",\"contentUrl\":\"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-d6127db92238e6e7d4a0a7b879f2cbd1.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/17aitech.com\/?p=35490#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/17aitech.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/17aitech.com\/#website\",\"url\":\"https:\/\/17aitech.com\/\",\"name\":\"\u4e00\u8d77AI\u6280\u672f\",\"description\":\"\u8ba9AI\u77e5\u8bc6\u89e6\u624b\u53ef\u53ca\",\"alternateName\":\"\u4e00\u8d77AI\u6280\u672f\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/17aitech.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/17aitech.com\/#\/schema\/person\/60225458499e817ae0af73e67e440b9d\",\"name\":\"AI\u5c0f\u52a9\u624b\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\/\/17aitech.com\/#\/schema\/person\/image\/\",\"url\":\"\/\/17aitech.com\/wp-content\/uploads\/2024\/04\/robot_3.png\",\"contentUrl\":\"\/\/17aitech.com\/wp-content\/uploads\/2024\/04\/robot_3.png\",\"caption\":\"AI\u5c0f\u52a9\u624b\"},\"description\":\"\u8fd9\u4e2a\u4eba\u5f88\u61d2\uff0c\u4ec0\u4e48\u90fd\u6ca1\u6709\u7559\u4e0b\uff5e\",\"url\":\"https:\/\/17aitech.com\/?page_id=33738&user=3\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3 - \u4e00\u8d77AI\u6280\u672f","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/17aitech.com\/?p=35490","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/17aitech.com\/?p=35490","url":"https:\/\/17aitech.com\/?p=35490","name":"\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3 - \u4e00\u8d77AI\u6280\u672f","isPartOf":{"@id":"https:\/\/17aitech.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/17aitech.com\/?p=35490#primaryimage"},"image":{"@id":"https:\/\/17aitech.com\/?p=35490#primaryimage"},"thumbnailUrl":"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-d6127db92238e6e7d4a0a7b879f2cbd1.png","datePublished":"2024-12-25T15:00:02+00:00","author":{"@id":"https:\/\/17aitech.com\/#\/schema\/person\/60225458499e817ae0af73e67e440b9d"},"breadcrumb":{"@id":"https:\/\/17aitech.com\/?p=35490#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/17aitech.com\/?p=35490"]}]},{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/17aitech.com\/?p=35490#primaryimage","url":"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-d6127db92238e6e7d4a0a7b879f2cbd1.png","contentUrl":"https:\/\/17aitech.com\/wp-content\/uploads\/2024\/09\/frc-d6127db92238e6e7d4a0a7b879f2cbd1.png"},{"@type":"BreadcrumbList","@id":"https:\/\/17aitech.com\/?p=35490#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/17aitech.com\/"},{"@type":"ListItem","position":2,"name":"\u8fd9\u5c31\u7ffb\u8f66\u4e86\uff1fReflection 70B\u906d\u8d28\u7591\u57fa\u6a21\u4e3aLlama 3\uff0c\u4f5c\u8005\uff1a\u91cd\u65b0\u8bad\u7ec3"}]},{"@type":"WebSite","@id":"https:\/\/17aitech.com\/#website","url":"https:\/\/17aitech.com\/","name":"\u4e00\u8d77AI\u6280\u672f","description":"\u8ba9AI\u77e5\u8bc6\u89e6\u624b\u53ef\u53ca","alternateName":"\u4e00\u8d77AI\u6280\u672f","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/17aitech.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/17aitech.com\/#\/schema\/person\/60225458499e817ae0af73e67e440b9d","name":"AI\u5c0f\u52a9\u624b","image":{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/17aitech.com\/#\/schema\/person\/image\/","url":"\/\/17aitech.com\/wp-content\/uploads\/2024\/04\/robot_3.png","contentUrl":"\/\/17aitech.com\/wp-content\/uploads\/2024\/04\/robot_3.png","caption":"AI\u5c0f\u52a9\u624b"},"description":"\u8fd9\u4e2a\u4eba\u5f88\u61d2\uff0c\u4ec0\u4e48\u90fd\u6ca1\u6709\u7559\u4e0b\uff5e","url":"https:\/\/17aitech.com\/?page_id=33738&user=3"}]}},"_links":{"self":[{"href":"https:\/\/17aitech.com\/index.php?rest_route=\/wp\/v2\/posts\/35490","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/17aitech.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/17aitech.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/17aitech.com\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/17aitech.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=35490"}],"version-history":[{"count":0,"href":"https:\/\/17aitech.com\/index.php?rest_route=\/wp\/v2\/posts\/35490\/revisions"}],"wp:attachment":[{"href":"https:\/\/17aitech.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=35490"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/17aitech.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=35490"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/17aitech.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=35490"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}