{"id":686,"date":"2025-01-29T07:35:31","date_gmt":"2025-01-29T07:35:31","guid":{"rendered":"https:\/\/janusai.pro\/?p=686"},"modified":"2025-01-29T07:37:05","modified_gmt":"2025-01-29T07:37:05","slug":"i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive","status":"publish","type":"post","link":"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/","title":{"rendered":"Znalosti DeepSeek-R1 o schopnostech uva\u017eov\u00e1n\u00ed jsem destiloval do Qwen2 a v\u00fdsledky byly opravdu v\u00fdbu\u0161n\u00e9!!!"},"content":{"rendered":"<div style=\"margin-top: 0px; margin-bottom: 0px;\" class=\"sharethis-inline-share-buttons\" ><\/div>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Obsah<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"P\u0159epnut\u00ed tabulky obsahu\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">P\u0159ep\u00edna\u010d<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#%E2%85%A0_What_is_knowledge_distillation\" >\u2160. Co je to destilace znalost\u00ed?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#IICore_concepts\" >II.Z\u00e1kladn\u00ed pojmy<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#21_Template_design\" >2.1 N\u00e1vrh \u0161ablony<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#22_Reasoning_trajectory_The_%E2%80%9Cthinking_chain%E2%80%9D_of_the_models_solution\" >2.2 Trajektorie uva\u017eov\u00e1n\u00ed: \"My\u0161lenkov\u00fd \u0159et\u011bzec\" \u0159e\u0161en\u00ed modelu<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#23_Rejection_sampling_Filtering_good_data_from_%E2%80%9Ctrial_and_error\" >2.3 Odb\u011br vzork\u016f: Filtrov\u00e1n\u00ed dobr\u00fdch dat z \"pokus\u016f a omyl\u016f<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#%E2%85%A2Generation_of_distilled_data\" >\u2162.Generov\u00e1n\u00ed destilovan\u00fdch dat<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#Data_sources\" >Zdroje dat:<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#Distillation_data_generation_process\" >Proces generov\u00e1n\u00ed destila\u010dn\u00edch dat:<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#%E2%85%A3Distillation_process\" >\u2163.Destila\u010dn\u00ed proces<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#Teacher_and_student_roles\" >Role u\u010ditele a \u017e\u00e1ka:<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#Training_steps\" >Kroky \u0161kolen\u00ed:<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#%E2%85%A4_Example_demonstration\" >\u2164. P\u0159\u00edklad demonstrace<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/janusai.pro\/cs\/i-distilled-deepseek-r1s-reasoning-ability-knowledge-into-qwen2-and-the-results-were-really-explosive\/#%E2%85%A5_Summary\" >\u2165. Shrnut\u00ed<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%E2%85%A0_What_is_knowledge_distillation\"><\/span><strong>\u2160. <\/strong>Co je destilace znalost\u00ed?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Destilace znalost\u00ed je technika komprese modelu, kter\u00e1 se pou\u017e\u00edv\u00e1 k p\u0159enosu znalost\u00ed z velk\u00e9ho, komplexn\u00edho modelu (model u\u010ditele) do mal\u00e9ho modelu (model studenta). <\/p>\n\n\n\n<p>Z\u00e1kladn\u00ed princip spo\u010d\u00edv\u00e1 v tom, \u017ee u\u010ditelsk\u00fd model u\u010d\u00ed \u017e\u00e1kovsk\u00fd model p\u0159edpov\u00edd\u00e1n\u00edm v\u00fdsledk\u016f (nap\u0159\u00edklad rozd\u011blen\u00ed pravd\u011bpodobnosti nebo odvozovac\u00edch proces\u016f) a \u017e\u00e1kovsk\u00fd model zlep\u0161uje sv\u016fj v\u00fdkon u\u010den\u00edm se z t\u011bchto p\u0159edpov\u011bd\u00ed. <\/p>\n\n\n\n<p>Tato metoda je vhodn\u00e1 zejm\u00e9na pro za\u0159\u00edzen\u00ed s omezen\u00fdmi zdroji, jako jsou mobiln\u00ed telefony nebo vestav\u011bn\u00e1 za\u0159\u00edzen\u00ed.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"IICore_concepts\"><\/span>II.Z\u00e1kladn\u00ed pojmy<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"21_Template_design\"><\/span>2.1 N\u00e1vrh \u0161ablony<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u0160ablona: \u0160ablona: strukturovan\u00fd form\u00e1t pou\u017e\u00edvan\u00fd ke standardizaci v\u00fdstupu modelu. Nap\u0159\u00edklad\n<ul class=\"wp-block-list\">\n<li>: Ozna\u010duje za\u010d\u00e1tek procesu uva\u017eov\u00e1n\u00ed.<\/li>\n\n\n\n<li>: Ozna\u010duje konec procesu uva\u017eov\u00e1n\u00ed.<\/li>\n\n\n\n<li>: Ozna\u010duje za\u010d\u00e1tek kone\u010dn\u00e9 odpov\u011bdi.<\/li>\n\n\n\n<li>: Ozna\u010duje konec z\u00e1v\u011bre\u010dn\u00e9 odpov\u011bdi.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Funkce:\n<ul class=\"wp-block-list\">\n<li>Jasnost: Stejn\u011b jako \"podn\u011btn\u00e1 slova\" v ot\u00e1zce na vypln\u011bn\u00ed pr\u00e1zdn\u00e9ho pol\u00ed\u010dka \u0159\u00edk\u00e1 modelu \"proces p\u0159em\u00fd\u0161len\u00ed prob\u00edh\u00e1 zde a odpov\u011b\u010f prob\u00edh\u00e1 zde\".<\/li>\n\n\n\n<li>D\u016fslednost: Zaji\u0161\u0165uje, \u017ee v\u0161echny v\u00fdstupy maj\u00ed stejnou strukturu, co\u017e usnad\u0148uje n\u00e1sledn\u00e9 zpracov\u00e1n\u00ed a anal\u00fdzu.<\/li>\n\n\n\n<li>\u010citelnost: Lid\u00e9 mohou snadno rozli\u0161it proces uva\u017eov\u00e1n\u00ed a odpov\u011b\u010f, co\u017e zlep\u0161uje u\u017eivatelsk\u00fd z\u00e1\u017eitek.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"22_Reasoning_trajectory_The_%E2%80%9Cthinking_chain%E2%80%9D_of_the_models_solution\"><\/span>2.2 Trajektorie uva\u017eov\u00e1n\u00ed: \"My\u0161lenkov\u00fd \u0159et\u011bzec\" \u0159e\u0161en\u00ed modelu<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Trajektorie uva\u017eov\u00e1n\u00ed: Podrobn\u00e9 kroky generovan\u00e9 modelem p\u0159i \u0159e\u0161en\u00ed probl\u00e9mu ukazuj\u00ed logick\u00fd \u0159et\u011bzec modelu.<\/li>\n\n\n\n<li>P\u0159\u00edklad:<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"759\" height=\"290\" src=\"https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/b8eff676-f9d7-436c-9ee7-1e423242825d.png\" alt=\"\" class=\"wp-image-689\" srcset=\"https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/b8eff676-f9d7-436c-9ee7-1e423242825d.png 759w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/b8eff676-f9d7-436c-9ee7-1e423242825d-300x115.png 300w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/b8eff676-f9d7-436c-9ee7-1e423242825d-18x7.png 18w\" sizes=\"auto, (max-width: 759px) 100vw, 759px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"23_Rejection_sampling_Filtering_good_data_from_%E2%80%9Ctrial_and_error\"><\/span>2.3 Odb\u011br vzork\u016f: Filtrov\u00e1n\u00ed dobr\u00fdch dat z \"pokus\u016f a omyl\u016f<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Odb\u011br vzork\u016f pro odm\u00edtnut\u00ed: Generov\u00e1n\u00ed v\u00edce kandid\u00e1tsk\u00fdch odpov\u011bd\u00ed a ponech\u00e1n\u00ed t\u011bch dobr\u00fdch, podobn\u011b jako p\u0159i psan\u00ed n\u00e1vrhu a n\u00e1sledn\u00e9m kop\u00edrov\u00e1n\u00ed spr\u00e1vn\u00e9 odpov\u011bdi p\u0159i zkou\u0161ce.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%E2%85%A2Generation_of_distilled_data\"><\/span>\u2162.Generov\u00e1n\u00ed destilovan\u00fdch dat<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Prvn\u00edm krokem p\u0159i destilaci znalost\u00ed je vytvo\u0159en\u00ed vysoce kvalitn\u00edch \"v\u00fdukov\u00fdch dat\", ze kter\u00fdch se budou mal\u00e9 modely u\u010dit.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Data_sources\"><\/span><strong>Zdroje dat<\/strong>:<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>80% z argumenta\u010dn\u00edch dat vygenerovan\u00fdch pomoc\u00ed <a href=\"https:\/\/huggingface.co\/deepseek-ai\/DeepSeek-R1\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">DeepSeek-R1<\/a><\/li>\n\n\n\n<li>20% z obecn\u00fdch dat \u00falohy DeepSeek-V3.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Distillation_data_generation_process\"><\/span><strong>Proces generov\u00e1n\u00ed destila\u010dn\u00edch dat<\/strong>:<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Filtrov\u00e1n\u00ed pravidel<\/strong>: automaticky kontroluje spr\u00e1vnost odpov\u011bdi (nap\u0159. zda matematick\u00e1 odpov\u011b\u010f odpov\u00edd\u00e1 vzorci).<\/li>\n\n\n\n<li><strong>Kontrola \u010ditelnosti<\/strong>: eliminuje sm\u00ed\u0161en\u00e9 jazyky (nap\u0159. sm\u00ed\u0161enou \u010d\u00edn\u0161tinu a angli\u010dtinu) nebo dlouh\u00e9 odstavce.<\/li>\n\n\n\n<li><strong>Generov\u00e1n\u00ed \u0159\u00edzen\u00e9 \u0161ablonou<\/strong>: vy\u017eaduje, aby DeepSeek-R1 vytv\u00e1\u0159el inferen\u010dn\u00ed trajektorie podle \u0161ablony.<\/li>\n\n\n\n<li><strong>Filtrov\u00e1n\u00ed s v\u00fdb\u011brem vzork\u016f<\/strong>:<\/li>\n\n\n\n<li><strong>Integrace dat<\/strong>: nakonec bylo vygenerov\u00e1no 800 000 vysoce kvalitn\u00edch vzork\u016f, z toho asi 600 000 inferen\u010dn\u00edch dat a asi 200 000 obecn\u00fdch dat.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%E2%85%A3Distillation_process\"><\/span>\u2163.Destila\u010dn\u00ed proces<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Teacher_and_student_roles\"><\/span>Role u\u010ditele a \u017e\u00e1ka:<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>DeepSeek-R1 jako model u\u010ditele;<\/li>\n\n\n\n<li>Modely \u0159ady Qwen jako studentsk\u00fd model.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Training_steps\"><\/span>Kroky \u0161kolen\u00ed:<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Nejprve vstup dat: do modelu Qwen je t\u0159eba zadat \u010d\u00e1st ot\u00e1zky z 800 000 vzork\u016f a po\u017e\u00e1dat jej o vygenerov\u00e1n\u00ed kompletn\u00ed inferen\u010dn\u00ed trajektorie (proces my\u0161len\u00ed + odpov\u011b\u010f) podle \u0161ablony. To je velmi d\u016fle\u017eit\u00fd krok<\/p>\n\n\n\n<p>D\u00e1le v\u00fdpo\u010det ztr\u00e1ty: porovn\u00e1n\u00ed v\u00fdstupu generovan\u00e9ho \u017e\u00e1kovsk\u00fdm modelem s inferen\u010dn\u00ed trajektori\u00ed u\u010ditelsk\u00e9ho modelu a zarovn\u00e1n\u00ed textov\u00e9 sekvence pomoc\u00ed dola\u010fov\u00e1n\u00ed pod dohledem (SFT). Pokud si nejste jisti, co je to SFT, douf\u00e1m, \u017ee vyhled\u00e1te toto kl\u00ed\u010dov\u00e9 slovo, abyste se dozv\u011bd\u011bli v\u00edce.<\/p>\n\n\n\n<p>Dokon\u010den\u00ed aktualizace parametr\u016f v\u011bt\u0161\u00edho modelu studenta: Optimalizujte parametry modelu Qwen pomoc\u00ed zp\u011btn\u00e9ho \u0161\u00ed\u0159en\u00ed, abyste aproximovali v\u00fdstup modelu u\u010ditele.<\/p>\n\n\n\n<p>N\u011bkolikan\u00e1sobn\u00e9 opakov\u00e1n\u00ed tohoto procesu \u0161kolen\u00ed zaji\u0161\u0165uje dostate\u010dn\u00fd p\u0159enos znalost\u00ed. T\u00edm je dosa\u017eeno p\u016fvodn\u00edho c\u00edle \u0161kolen\u00ed. Uvedeme v\u00e1m p\u0159\u00edklad, na kter\u00e9m to demonstrujeme, a douf\u00e1me, \u017ee to pochop\u00edte.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%E2%85%A4_Example_demonstration\"><\/span>\u2164. P\u0159\u00edklad demonstrace<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>\u010cl\u00e1nek demonstruje destila\u010dn\u00ed efekt na konkr\u00e9tn\u00ed \u00faloze \u0159e\u0161en\u00ed rovnice (solve equation):<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Standardn\u00ed v\u00fdstup modelu u\u010ditele:<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"771\" height=\"328\" src=\"https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/3a53b6a8-36d2-4251-ab0f-8646d7646352.png\" alt=\"\" class=\"wp-image-690\" srcset=\"https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/3a53b6a8-36d2-4251-ab0f-8646d7646352.png 771w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/3a53b6a8-36d2-4251-ab0f-8646d7646352-300x128.png 300w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/3a53b6a8-36d2-4251-ab0f-8646d7646352-768x327.png 768w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/3a53b6a8-36d2-4251-ab0f-8646d7646352-18x8.png 18w\" sizes=\"auto, (max-width: 771px) 100vw, 771px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>V\u00fdstup Qwen-7B p\u0159ed destilac\u00ed:<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"766\" height=\"178\" src=\"https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/51c44a52-01a0-474a-8d47-5483613286fb.png\" alt=\"\" class=\"wp-image-688\" srcset=\"https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/51c44a52-01a0-474a-8d47-5483613286fb.png 766w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/51c44a52-01a0-474a-8d47-5483613286fb-300x70.png 300w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/51c44a52-01a0-474a-8d47-5483613286fb-18x4.png 18w\" sizes=\"auto, (max-width: 766px) 100vw, 766px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>V\u00fdstup Qwen-7B po destilaci:<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"759\" height=\"260\" src=\"https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/61c7fb80-d903-4339-971c-9613b5ac199c.png\" alt=\"\" class=\"wp-image-687\" srcset=\"https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/61c7fb80-d903-4339-971c-9613b5ac199c.png 759w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/61c7fb80-d903-4339-971c-9613b5ac199c-300x103.png 300w, https:\/\/janusai.pro\/wp-content\/uploads\/2025\/01\/61c7fb80-d903-4339-971c-9613b5ac199c-18x6.png 18w\" sizes=\"auto, (max-width: 759px) 100vw, 759px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Optimalizovan\u00e9 \u0159e\u0161en\u00ed: Odpov\u011b\u010f je stejn\u00e1 jako u modelu u\u010ditele.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%E2%85%A5_Summary\"><\/span>\u2165. Shrnut\u00ed<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Prost\u0159ednictv\u00edm destilace znalost\u00ed se schopnost odvozov\u00e1n\u00ed DeepSeek-R1 efektivn\u011b p\u0159en\u00e1\u0161\u00ed do \u0159ady mal\u00fdch model\u016f Qwen. Tento proces se zam\u011b\u0159uje na \u0161ablonovit\u00fd v\u00fdstup a vzorkov\u00e1n\u00ed odm\u00edtnut\u00ed. D\u00edky strukturovan\u00e9mu generov\u00e1n\u00ed dat a zdokonalen\u00e9mu tr\u00e9ninku mohou mal\u00e9 modely prov\u00e1d\u011bt slo\u017eit\u00e9 inferen\u010dn\u00ed \u00falohy i ve sc\u00e9n\u00e1\u0159\u00edch s omezen\u00fdmi zdroji. Tato technologie poskytuje d\u016fle\u017eitou referenci pro odleh\u010den\u00e9 nasazen\u00ed model\u016f um\u011bl\u00e9 inteligence.<\/p>","protected":false},"excerpt":{"rendered":"<p>\u2160. Co je to destilace znalost\u00ed? Destilace znalost\u00ed je technika komprese modelu, kter\u00e1 se pou\u017e\u00edv\u00e1 k p\u0159enosu znalost\u00ed z velk\u00e9ho, komplexn\u00edho modelu (model u\u010ditele) na mal\u00fd model (model studenta). Z\u00e1kladn\u00ed princip spo\u010d\u00edv\u00e1 v tom, \u017ee u\u010ditelsk\u00fd model u\u010d\u00ed studentsk\u00fd model p\u0159edpov\u00edd\u00e1n\u00edm v\u00fdsledk\u016f (nap\u0159\u00edklad rozd\u011blen\u00ed pravd\u011bpodobnosti nebo odvozovac\u00edch proces\u016f) a...<\/p>","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kadence_starter_templates_imported_post":false,"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-686","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/posts\/686","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/comments?post=686"}],"version-history":[{"count":2,"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/posts\/686\/revisions"}],"predecessor-version":[{"id":692,"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/posts\/686\/revisions\/692"}],"wp:attachment":[{"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/media?parent=686"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/categories?post=686"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/janusai.pro\/cs\/wp-json\/wp\/v2\/tags?post=686"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}