![\"\"](\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2023\/05\/Screenshot-2023-05-20-at-4.46.14-PM-1024x574.png\")
![\"\"](\"https:\/\/www.marktechpost.com\/wp-content\/uploads\/2023\/05\/Screenshot-2023-05-20-at-4.46.14-PM-150x150.png\")
n<\/p>\nn<\/p>\n
Modern large language models (LLMs) have excellent performance on code reading and generation tasks, allowing more people to enter the once-mysterious field of computer programming. Architecturally, existing code LLMs use encoder- or decoder-only models, which excel at just some comprehension and generating tasks. Code-focused LLMs typically have a limited set of pretraining objectives, which will degrade performance on downstream tasks that are less relevant to those objectives, and they often adopt an encoder-only or decoder-only architecture, which can limit their optimal performance to only specific tasks.<\/p>\n
nnnn<\/p>\n
The AI Research team at Salesforce presents CodeT5+. It is a revolutionary family of encoder-decoder code foundation LLMs that can be easily customized to perform exceptionally well on various code interpretation and generation tasks. To do this, the team provides CodeT5+ with a wide range of pretraining objectives on unimodal and bimodal data to provide a code LLM that can be easily adapted to various downstream tasks.<\/p>\n
nnnn<\/p>\n
What is CodeT5+<\/strong><\/p>\n nnnn<\/p>\n CodeT5+ is a set of large-scale language models for analyzing and generating code. The framework incorporates a wide range of unimodal and bimodal pretraining goals. CodeT5+’s modules can be separated and recombined flexibly to meet the needs of a wide variety of zero-shot, finetuning, and instruction-tuning applications.<\/p>\n nnnn<\/p>\n While the decoder is trained to provide various outputs based on the pretraining learning tasks, the encoder learns to encode contextual representations from code\/text sequences (entire, partial, or span-masked sequences).<\/p>\n nnnn<\/p>\n n<\/ul>\n nnnn<\/p>\n n<\/ul>\n nnnn<\/p>\n CodeT5+ can adapt its performance to various tasks thanks to its two-stage pretraining procedure, which includes seq2seq-generating tasks, decoder-only activities, and understanding-based tasks.<\/p>\n nnnn<\/p>\n In their empirical investigation, the team compared CodeT5+ against 20 benchmark datasets and state-of-the-art code LLMs, including LaMDA, GPT, StarCoder, etc., on tasks including zero-shot, finetuning, and instruction-tuning. While competing against OpenAI’s robust code-cushman-001 model, CodeT5+ achieved State-of-the-Art (SOTA) outcomes on zero-shot HumanEval code creation tasks.<\/p>\n nnnn<\/p>\n To sum it up<\/strong><\/p>\n nnnn<\/p>\n CodeT5+ is a new family of open-source, large-language models with an encoder-decoder architecture that may function in several modes (encoder-only, decoder-only, and encoder-decoder) to serve a variety of code interpretation and generation activities. CodeT5+ is trained using a variety of pretraining tasks, including span denoising, causal language modeling, contrastive learning, and text-code matching to acquire a comprehensive understanding of both unimodal and bimodal code-text data.<\/p>\n nnnn<\/p>\n This work indicates that the proposed CodeT5+ open code LLMs can support and even reach SOTA performance across a wide range of downstream code jobs by operating flexibly in encoder-only, decoder-only, and encoder-decoder modes. The team is open-sourcing all CodeT5+ models to encourage further study because they believe CodeTs+ can be deployed as a unified retrieval-augmented generation system.<\/p>\n nnnn<\/p>\n n Modern large language models (LLMs) have excellent performance on code reading and generation tasks, allowing more people to enter the once-mysterious field of computer programming. Architecturally, existing code LLMs use encoder- or decoder-only models, which excel at just some comprehension and generating tasks. Code-focused LLMs typically have a limited set of pretraining objectives, which […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-22120","post","type-post","status-publish","format-standard","hentry","category-news"],"_links":{"self":[{"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/posts\/22120","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/comments?post=22120"}],"version-history":[{"count":0,"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/posts\/22120\/revisions"}],"wp:attachment":[{"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/media?parent=22120"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/categories?post=22120"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/healthmedicinet.com\/business\/wp-json\/wp\/v2\/tags?post=22120"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}n<\/p>\n
n<\/p>\n
nnn n<\/p>\n","protected":false},"excerpt":{"rendered":"