{"id":746,"date":"2025-01-30T13:05:33","date_gmt":"2025-01-30T13:05:33","guid":{"rendered":"https:\/\/janusai.pro\/?p=746"},"modified":"2025-01-30T13:05:35","modified_gmt":"2025-01-30T13:05:35","slug":"the-complete-explanation-from-deepseek-janus-to-janus-pro","status":"publish","type":"post","link":"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/","title":{"rendered":"Explica\u021bia complet\u0103: de la DeepSeek Janus la Janus-Pro!"},"content":{"rendered":"<div style=\"margin-top: 0px; margin-bottom: 0px;\" class=\"sharethis-inline-share-buttons\" ><\/div>\n<p>Mesaj de luat acas\u0103: Janus este un model simplu, unificat \u0219i extensibil de \u00een\u021belegere \u0219i generare multimodal\u0103 care decupleaz\u0103 \u00een\u021belegerea multimodal\u0103 \u0219i codarea vizual\u0103 generat\u0103, atenu\u00e2nd poten\u021bialele conflicte dintre cele dou\u0103 sarcini. Acesta poate fi extins pentru a \u00eencorpora modalit\u0103\u021bi de intrare suplimentare \u00een viitor. Janus-Pro se bazeaz\u0103 pe aceast\u0103 funda\u021bie prin optimizarea strategiei de formare (inclusiv cre\u0219terea num\u0103rului de etape de formare, ajustarea rapoartelor de date etc.), ad\u0103ugarea mai multor date (inclusiv utilizarea de date sintetice etc.) \u0219i m\u0103rirea dimensiunii modelului (la 7 miliarde de parametri), ceea ce conduce la progrese \u00een capacit\u0103\u021bile de \u00een\u021belegere multimodal\u0103 \u0219i de respectare a instruc\u021biunilor text-imagine ale modelului.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/rmy9ct2fln.feishu.cn\/space\/api\/box\/stream\/download\/asynccode\/?code=Mjg4MjEwYjVlNzk0YTgyMTc0NDJlODQ4MTU2ZmRjYTVfWnhaaVEyZlEwUHFrUHNUeGNCOWpCRU1EVDN0QktBMUxfVG9rZW46SkVQZmJmSEhqb1g4YTJ4MVNYdmNPT2oybmVmXzE3MzgyNDIwMzc6MTczODI0NTYzN19WNA\" alt=\"\"\/><\/figure>\n\n\n\n<p><a href=\"https:\/\/github.com\/deepseek-ai\/JanusJanus\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Adresa codului<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/github.com\/deepseek-ai\/Janus\/blob\/main\/janus_pro_tech_report.pdf\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Adresa Janus Pro<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/huggingface.co\/deepseek-ai\/Janus-Pro-7B\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Janus-Pro<\/a> este o versiune avansat\u0103 a lucr\u0103rii anterioare Janus, \u00een special, incluz\u00e2nd (1) o strategie de formare optimizat\u0103, (2) date de formare extinse \u0219i (3) dimensiuni mai mari ale modelului. Cu aceste \u00eembun\u0103t\u0103\u021biri, Janus-Pro face progrese semnificative \u00een capacit\u0103\u021bile de \u00een\u021belegere multimodal\u0103 \u0219i de respectare a instruc\u021biunilor text-imagine, \u00eembun\u0103t\u0103\u021bind \u00een acela\u0219i timp stabilitatea gener\u0103rii text-imagine. \u00cenainte de a detalia Janus-Pro, s\u0103 trecem \u00een revist\u0103 Janus.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Tabla de con\u021binut<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Tabelul de con\u021binut\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Reviewing_Janus\" >Revizuirea Janus<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Janus_training_is_divided_into_3_phases\" >Formarea Janus este \u00eemp\u0103r\u021bit\u0103 \u00een 3 faze:<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Phase_1\" >Faza 1<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Phase_2\" >Faza 2<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Phase_3\" >Faza 3<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Training_Objectives\" >Obiective de formare<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Reasoning\" >Ra\u021bionament<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Possible_extensions\" >Extinderi posibile<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Janus-Pro_Upgrade\" >Actualizare Janus-Pro<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Main_Improvements\" >\u00cembun\u0103t\u0103\u021biri principale<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Training_Strategy\" >Strategia de formare<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Data_Scale\" >Scala de date<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Model_Scale\" >Model la scar\u0103<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Experimental_details\" >Detalii experimentale<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/janusai.pro\/ro\/the-complete-explanation-from-deepseek-janus-to-janus-pro\/#Insufficient\" >Insuficient<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Reviewing_Janus\"><\/span>Revizuirea Janus<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Predecesorul Janus este un cadru autoregresiv pentru \u00een\u021belegerea \u0219i generarea multimodal\u0103 unificat\u0103, care este utilizat pentru decuplarea codific\u0103rii vizuale pentru \u00een\u021belegerea \u0219i generarea multimodal\u0103 unificat\u0103. Pentru \u00een\u021belegerea multimodal\u0103, proiectarea urmeaz\u0103 de obicei LLaVA, folosind codificatorii vizuali ca o punte pentru a permite modelelor lingvistice mari s\u0103 \u00een\u021beleag\u0103 imaginile. Pentru generare, aceasta se bazeaz\u0103 de obicei pe modele de difuzie, iar unele se bazeaz\u0103 pe metode autoregresive. Unele abord\u0103ri \u00eencearc\u0103 s\u0103 utilizeze un singur transformator pentru a \u00eencerca s\u0103 unifice sarcinile de \u00een\u021belegere \u0219i generare multimodal\u0103, care utilizeaz\u0103 de obicei un singur codificator vizual pentru a procesa intr\u0103rile ambelor sarcini.<\/p>\n\n\n\n<p>Cu toate acestea, exist\u0103 diferen\u021be \u00een reprezent\u0103rile necesare pentru sarcinile de \u00een\u021belegere \u0219i generare multimodal\u0103. \u00cen sarcina de \u00een\u021belegere multimodal\u0103, codificatorul vizual urm\u0103re\u0219te s\u0103 extrag\u0103 informa\u021bii semantice de nivel \u00eenalt (de exemplu, categorii de obiecte sau atribute vizuale), iar ie\u0219irea implic\u0103 nu numai extragerea informa\u021biilor din imagine, ci \u0219i ra\u021bionamentul semantic complex, codificatorul concentr\u00e2ndu-se \u00een principal pe reprezent\u0103ri semantice \u00eenalt-dimensionale. Sarcina de generare se refer\u0103 \u00een principal la generarea detaliilor locale \u0219i la men\u021binerea coeren\u021bei globale a imaginii, necesit\u00e2nd astfel reprezent\u0103ri codate bidimensionale reduse ale structurilor spa\u021biale \u0219i ale detaliilor texturii. Unificarea reprezent\u0103rilor ambelor sarcini \u00een acela\u0219i spa\u021biu poate duce la conflicte.<\/p>\n\n\n\n<p>Janus con\u021bine 2 c\u0103i independente de codificare vizual\u0103 pentru \u00een\u021belegerea \u0219i generarea multimodal\u0103 \u0219i aduce dou\u0103 beneficii: 1) atenueaz\u0103 conflictele care decurg din cerin\u021bele diferite de granularitate ale \u00een\u021belegerii \u0219i gener\u0103rii multimodale \u0219i 2) este flexibil \u0219i scalabil, decuplabil, astfel \u00eenc\u00e2t at\u00e2t sarcinile de \u00een\u021belegere, c\u00e2t \u0219i cele de generare pot fi codificate utiliz\u00e2nd tehnici de codificare de ultim\u0103 or\u0103 specifice domeniilor lor, iar \u00een viitor pot fi alimentate cu nori de puncte, semnale EEG sau date audio \u0219i prelucrate utiliz\u00e2nd un transformator unificat.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/rmy9ct2fln.feishu.cn\/space\/api\/box\/stream\/download\/asynccode\/?code=OTE3ZjkyNWQ5MmUwNDQzM2VjN2VlNWYwZjAxYTVmZGRfMXpJMWVObDBKOHYxTVJqeEw2S0pHT2hGU3RuVHdnWVdfVG9rZW46UDQyQ2Jrb0Myb1h0bjR4TFBrV2NRS29GbkRmXzE3MzgyNDIwMzc6MTczODI0NTYzN19WNA\" alt=\"\"\/><\/figure>\n\n\n\n<p>Pentru \u00een\u021belegerea textului, textul este convertit \u00een ID-uri discrete utiliz\u00e2nd Tokenizer-ul \u00eencorporat al LLM;<\/p>\n\n\n\n<p>Pentru \u00een\u021belegerea multimodal\u0103, caracteristicile semantice \u00eenalt-dimensionale din imagini sunt extrase utiliz\u00e2nd codificatoare SigLIP (nota autorului: Cosmos utilizeaz\u0103, de asemenea, codificatoare SigLIP \u00een sec\u021biunea Guardrails), iar caracteristicile extrase sunt mapate \u00een spa\u021biul caracteristicilor textului LLM utiliz\u00e2nd Adaptor (MLP cu 2 straturi);<\/p>\n\n\n\n<p>Partea lung\u0103 a fost ajustat\u0103 la 384 de pixeli, iar partea scurt\u0103 a fost umplut\u0103 la 384 de pixeli folosind RGB(127, 127, 127);<\/p>\n\n\n\n<p>Pentru generarea vizual\u0103, imaginea a fost convertit\u0103 \u00een ID-uri discrete utiliz\u00e2nd VQ Tokenizer, iar fiecare ID a fost mapat \u00een spa\u021biul caracteristic textual al LLM utiliz\u00e2nd Adaptor (MLP cu 2 straturi);<\/p>\n\n\n\n<p>Marginile scurte au fost redimensionate la 384 pixeli, iar marginile lungi au fost decupate la 384 pixeli;<\/p>\n\n\n\n<p>Preg\u0103tirea general\u0103 a fost efectuat\u0103 utiliz\u00e2nd 16 noduri, fiecare con\u021bin\u00e2nd 8 GPU Nvidia A100;<\/p>\n\n\n\n<p>At\u00e2t pentru generarea vizual\u0103, c\u00e2t \u0219i pentru sarcinile de \u00een\u021belegere multimodal\u0103, secven\u021bele de caracteristici ale imaginii \u0219i secven\u021bele de caracteristici ale textului sunt legate \u00eempreun\u0103 ca intrare pentru LLM (DeepSeek-LLM 1.3B este utilizat \u00een text);<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Capul de predic\u021bie \u00eencorporat al LLM este utilizat pentru predic\u021biile de text at\u00e2t \u00een sarcinile de \u00een\u021belegere a textului pur, c\u00e2t \u0219i \u00een cele de \u00een\u021belegere multimodal\u0103, \u00een timp ce un cap de predic\u021bie ini\u021bializat aleatoriu este utilizat pentru predic\u021biile de imagine \u00een sarcina de generare vizual\u0103. \u00centregul model ader\u0103 la un cadru autoregresiv f\u0103r\u0103 a fi nevoie de m\u0103\u0219ti de aten\u021bie special concepute.<\/p>\n<\/blockquote>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Janus_training_is_divided_into_3_phases\"><\/span><a href=\"https:\/\/huggingface.co\/deepseek-ai\/Janus-Pro-7B\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Formare Janus<\/a> este \u00eemp\u0103r\u021bit \u00een 3 faze:<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Phase_1\"><\/span>Faza 1<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p><strong>Adaptor de tren \u0219i cap de imagine<\/strong> s\u0103 creeze conexiuni \u00eentre elementele lingvistice \u0219i vizuale \u00een spa\u021biul de \u00eencorporare, permi\u021b\u00e2nd LLM s\u0103 \u00een\u021beleag\u0103 entit\u0103\u021bile din imagine \u0219i s\u0103 aib\u0103 capacit\u0103\u021bi ini\u021biale de generare vizual\u0103;<\/p>\n\n\n\n<p>Pentru \u00een\u021belegerea multimodal\u0103, folosi\u021bi 1,25 milioane de date cu didascalii \u00eemperecheate imagine-text din SHareGPT4V \u00een formatul: ;<\/p>\n\n\n\n<p>Pentru generarea vizual\u0103, folosind 1,2 milioane de probe din ImageNet1k \u00een formatul: ;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Phase_2\"><\/span>Faza 2<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p><strong>Preinstruire unificat\u0103<\/strong>, folosind un corpus multimodal pentru o pre-antrenare unificat\u0103 pentru a \u00eenv\u0103\u021ba \u00een\u021belegerea \u0219i generarea multimodal\u0103. \u00cen aceast\u0103 faz\u0103 sunt utilizate date de text simplu, date de \u00een\u021belegere multimodal\u0103 \u0219i date de generare vizual\u0103. Formare simpl\u0103 pentru generarea vizual\u0103 utiliz\u00e2nd ImageNet-1k, urmat\u0103 de utilizarea datelor generice text-imagine pentru a \u00eembun\u0103t\u0103\u021bi generarea vizual\u0103 \u00een domeniul deschis al modelului;<\/p>\n\n\n\n<p>Date text simplu: Corpus pre-antrenat DeepSeek-LLM;<\/p>\n\n\n\n<p>Date intercalate imagine-text: Seturile de date WikiHow \u0219i WIT;<\/p>\n\n\n\n<p>Date privind legendele imaginilor: Imagini din mai multe surse \u0219i redenumirea unora dintre imagini folosind modele multimodale din surse deschise, cu date formatate ca perechi Q&amp;A, de exemplu Denume\u0219te imaginea \u00een detaliu.<caption>;<\/p>\n\n\n\n<p>Date tabulare \u0219i grafice: date tabulare \u0219i grafice corespunz\u0103toare din DeepSeek-VL \u00een formatul ;<\/p>\n\n\n\n<p>Date generate vizual: perechi imagine-capitol din mai multe seturi de date \u0219i 2 milioane de date interne;<\/p>\n\n\n\n<p>\u00cen timpul antrenamentului, doar prima propozi\u021bie din legend\u0103 este utilizat\u0103 aleatoriu cu o probabilitate de 25%;<\/p>\n\n\n\n<p>E\u0219antioanele ImageNet apar doar \u00een primii 120K pa\u0219i de formare, iar imaginile din alte seturi de date apar \u00een urm\u0103torii 60K pa\u0219i;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Phase_3\"><\/span>Faza 3<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p><strong>Reglare fin\u0103 supravegheat\u0103<\/strong>, \u00een care modelele pre-antrenate sunt ajustate cu ajutorul datelor de ajustare a instruc\u021biunilor pentru a spori capacitatea acestora de a urma instruc\u021biunile \u0219i dialogul. Reglarea fin\u0103 a tuturor parametrilor, cu excep\u021bia codificatorului generator. Mascarea indica\u021biilor sistemului \u0219i ale utilizatorului \u00een timpul supravegherii r\u0103spunsurilor. Pentru a ne asigura c\u0103 Janus are competen\u021b\u0103 at\u00e2t \u00een \u00een\u021belegerea multimodal\u0103, c\u00e2t \u0219i \u00een generare, modelele nu sunt ajustate separat pentru sarcini specifice. \u00cen schimb, folosim un amestec de date de dialog numai text, date de \u00een\u021belegere multimodal\u0103 \u0219i date de generare vizual\u0103 pentru a asigura versatilitatea \u00eentr-o varietate de scenarii;<\/p>\n\n\n\n<p>\u00cen\u021belegerea textului: utilizeaz\u0103 date din surse specifice;<\/p>\n\n\n\n<p>\u00cen\u021belegerea multimodal\u0103: utilizarea datelor din surse multiple pentru reglarea instruc\u021biei;<\/p>\n\n\n\n<p>Generarea vizual\u0103: utiliz\u00e2nd un subset de perechi imagine-text din unele seturi de date din faza II, precum \u0219i 4 milioane de date interne;<\/p>\n\n\n\n<p>Formatul datelor este: Utilizator: \\n Asistent: ;<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/rmy9ct2fln.feishu.cn\/space\/api\/box\/stream\/download\/asynccode\/?code=M2I3MWQ5MjQyNTM5NjIyZTkyMjdlODgwMDg5NzIwYzJfSGVTUnVzb0I3bEREQXBkMEJGN0lqT0JBaEVUWEQwS05fVG9rZW46Vm9OMWJzYnNsbzRGR1R4YlJrNWNad1psblhjXzE3MzgyNDIwMzc6MTczODI0NTYzN19WNA\" alt=\"\"\/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Training_Objectives\"><\/span>Obiective de formare<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Janus este un model autoregresiv antrenat cu ajutorul unei func\u021bii de pierdere de entropie \u00eencruci\u0219at\u0103, pentru sarcinile de \u00een\u021belegere a textului simplu \u0219i de \u00een\u021belegere multimodal\u0103 pierderea fiind calculat\u0103 la secven\u021ba de text. Pentru sarcinile de generare vizual\u0103, pierderea este calculat\u0103 numai la secven\u021ba de imagini. Pentru a p\u0103stra simplitatea proiectului, nu sunt atribuite ponderi de pierdere diferite pentru diferitele sarcini.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Reasoning\"><\/span>Ra\u021bionament<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Folosind urm\u0103toarea metod\u0103 de predic\u021bie a elementelor lexicale, pentru \u00een\u021belegerea textului simplu \u0219i a \u00een\u021belegerii multimodale, elementele lexicale sunt e\u0219antionate secven\u021bial din distribu\u021bia de predic\u021bie. Pentru generarea imaginilor, se utilizeaz\u0103 un bootstrap f\u0103r\u0103 clasificator.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Possible_extensions\"><\/span>Extinderi posibile<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Pentru \u00een\u021belegerea multimodal\u0103, 1) ar putea fi ales un codificator vizual mai puternic \u0219i 2) ar putea fi utilizate tehnici dinamice de \u00eenalt\u0103 rezolu\u021bie;<\/p>\n\n\n\n<p>Pentru generarea vederii, 1) ar putea fi ale\u0219i codificatori mai fini, 2) utilizarea func\u021biilor de pierdere concepute special pentru generarea vederii \u0219i 3) combinarea aten\u021biei cauzale \u0219i a metodelor paralele;<\/p>\n\n\n\n<p>Mai multe modalit\u0103\u021bi, cu capacitatea de a integra nori de puncte 3D, haptic\u0103, EEG \u0219i alte intr\u0103ri pentru modalit\u0103\u021bile de pierdere;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Janus-Pro_Upgrade\"><\/span><a href=\"https:\/\/huggingface.co\/deepseek-ai\/Janus-Pro-7B\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Actualizare Janus-Pro<\/a><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Cu date de instruire limitate \u0219i o capacitate relativ mic\u0103 a modelului (1B), Janus este deficitar \u00een unele aspecte, cum ar fi reprezentarea slab\u0103 a gener\u0103rii de imagini sub indicii scurte \u0219i calitatea inconsistent\u0103 a gener\u0103rii textului \u00een imagine.Arhitectura Janus-Pro este aceea\u0219i cu cea a lui Janus, care poate fi v\u0103zut\u0103 \u00een figura de mai jos:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/rmy9ct2fln.feishu.cn\/space\/api\/box\/stream\/download\/asynccode\/?code=NDY0ZWM0NTJiOTNlYTE4MWI4NmMwNGE4Mjc3NmYyMDJfc1FEMHVOMHo1OUM0ZVhoakJtU1lZQXdZNTd4NVFXRzhfVG9rZW46RjJrTGI3VVlqb0IxS3N4aHVVN2NxUWxJbnZkXzE3MzgyNDIwMzc6MTczODI0NTYzN19WNA\" alt=\"\"\/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Main_Improvements\"><\/span>\u00cembun\u0103t\u0103\u021biri principale<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<h4 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Training_Strategy\"><\/span>Strategia de formare<span class=\"ez-toc-section-end\"><\/span><\/h4>\n\n\n\n<p>Etapa 1: Cre\u0219terea num\u0103rului de etape de formare \u0219i formarea complet\u0103 pe ImageNet;<\/p>\n\n\n\n<p>Etapa 2: Nu se mai utilizeaz\u0103 ImageNet, se utilizeaz\u0103 direct date obi\u0219nuite text-imagine pentru instruire;<\/p>\n\n\n\n<p>Etapa 3: Modificarea raporturilor seturilor de date \u00een procesul de reglare fin\u0103 prin schimbarea raportului dintre datele multimodale, datele text simplu \u0219i datele text-imagine de la 7:3:10 la 5:1:4;<\/p>\n\n\n\n<h4 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Data_Scale\"><\/span>Scala de date<span class=\"ez-toc-section-end\"><\/span><\/h4>\n\n\n\n<p>\u00cen\u021belegerea multimodal\u0103<\/p>\n\n\n\n<p>Etapa 2: Ad\u0103ugarea a 90 de milioane de mostre, inclusiv YFCC pentru subtitrarea imaginilor \u0219i Doc-matrix pentru \u00een\u021belegerea documentelor tabelare \u0219i grafice;<\/p>\n\n\n\n<p>Etapa 3: Ad\u0103ugarea seturilor de date suplimentare DeepSeek-VL2, cum ar fi \u00een\u021belegerea MEME;<\/p>\n\n\n\n<p>Generarea vizual\u0103: datele din lumea real\u0103 pot avea o calitate slab\u0103, ceea ce duce la o generare instabil\u0103 a textului \u00een imagine \u0219i la un rezultat estetic slab, Janus-Pro utilizeaz\u0103 72 de milioane de e\u0219antioane de date estetice sintetice, cu o faz\u0103 uniform\u0103 de preantrenare (etapa 2) de 1:1 raport \u00eentre datele reale \u0219i datele sintetice;<\/p>\n\n\n\n<h4 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Model_Scale\"><\/span>Model la scar\u0103<span class=\"ez-toc-section-end\"><\/span><\/h4>\n\n\n\n<p>Scala parametrilor modelului la o scar\u0103 de 7 miliarde de parametri;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Experimental_details\"><\/span>Detalii experimentale<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Comparativ cu Janus, detaliile experimentelor Janus-Pro sunt practic acelea\u0219i. \u00cen schimb, modelul cu parametri mai mari a utilizat mai multe noduri de cluster (de la 16 la 32).<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/rmy9ct2fln.feishu.cn\/space\/api\/box\/stream\/download\/asynccode\/?code=NDM1YTM1ZDliNDUwYzAzNzg4MTNiNjUzYWZlZjVhZjhfZGI5ZWloREhYV29OZUxiaEVFc0dhN1dMTDhGdG5ZSnNfVG9rZW46STA0amJtbVlhb0NySk94NkRKNmNqNDVybmdiXzE3MzgyNDIwMzc6MTczODI0NTYzN19WNA\" alt=\"\"\/><\/figure>\n\n\n\n<p>Janus-Pro hiperparametri de formare<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Insufficient\"><\/span>Insuficient<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Pentru \u00een\u021belegerea multimodal\u0103, rezolu\u021bia de intrare este limitat\u0103 la 384\u00d7384, afect\u00e2nd performan\u021ba \u00een sarcinile vizuale fine. Pentru generarea text-imagine, rezolu\u021bia redus\u0103 duce la o lips\u0103 de detalii \u00een rezultatele generate.<\/p>","protected":false},"excerpt":{"rendered":"<p>Mesaj de luat acas\u0103: Janus este un model simplu, unificat \u0219i extensibil de \u00een\u021belegere \u0219i generare multimodal\u0103 care decupleaz\u0103 \u00een\u021belegerea multimodal\u0103 \u0219i codarea vizual\u0103 generat\u0103, atenu\u00e2nd poten\u021bialele conflicte dintre cele dou\u0103 sarcini. Acesta poate fi extins pentru a \u00eencorpora modalit\u0103\u021bi de intrare suplimentare \u00een viitor. Janus-Pro se bazeaz\u0103 pe aceast\u0103 funda\u021bie prin optimizarea strategiei de formare (inclusiv cre\u0219terea...<\/p>","protected":false},"author":2,"featured_media":684,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kadence_starter_templates_imported_post":false,"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-746","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/posts\/746","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/comments?post=746"}],"version-history":[{"count":1,"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/posts\/746\/revisions"}],"predecessor-version":[{"id":747,"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/posts\/746\/revisions\/747"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/media\/684"}],"wp:attachment":[{"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/media?parent=746"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/categories?post=746"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/janusai.pro\/ro\/wp-json\/wp\/v2\/tags?post=746"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}