Large Language Model (LLM) "" AI (prompt)LLM LLM LLMChatGLM-6B ChatGLM-6B General Language Model (GLM) 62 INT4 6GB ChatGLM-6B ChatGLM 1T 62 ChatGLM-6B VisualGLM-6B VisualGLM-6B ChatGLM-6B 62 BLIP2-Qformer 78MOSS MOSS moss-moon 160 FP16 A100/A800 3090 INT4/8 3090 MOSS DB-GPT DB-GPT GPT GPT 100% 100% DB-GPT CPM-Bee CPM-Bee Transformer auto-regressiveCPM-Bee OpenBMB CPM-Bee CPM-Bee CPM-Bee OpenBMB OpenBMB CPM-Bee OpenBMB CPM-Bee CPM-Bee LaWGPT LaWGPT Chinese-LLaMAChatGLM (Linly) 32*A100 GPU baseline33B Linly-Chinese-LLAMA LLaMA CUDA CPU Huggingface Linly-Chinese-LLaMA LLaMA 7B13B 33B 65B Linly-ChatFlow 400 7B13B Linly-ChatFlow-int4 ChatFlow 4-bit CPU Linly-Chinese-BLOOM BLOOM 7B 175B Chinese-Vicuna LLaMA Chinese-Vicuna LLaMA+Lora finetune CPU ( C++) / / Facebook llama.ckpt Chinese-LLaMA-Alpaca LLaMA & Alpaca Chinese-LLaMA-Alpaca LLaMA Alpaca LLaMA Alpaca ChatYuan ChatYuan ChatYuan-large-v2 v1 ChatYuan-large-v2 ChatYuan PC INT4 400M GPT HuatuoGPT GPT ChatGPT HuatuoGPT ChatGPT LLaMA (BenTsao) (HuaTuo) LLaMA / (Instruct-tuning) LLaMA-7B GPT3

Large Language Model (LLM)

"" AI (prompt)

LLM

LLM

LLM

ChatGLM-6B

ChatGLM-6B General Language Model (GLM) 62 INT4 6GB

ChatGLM-6B ChatGLM 1T 62 ChatGLM-6B

VisualGLM-6B

VisualGLM-6B ChatGLM-6B 62 BLIP2-Qformer 78

MOSS

MOSS moss-moon 160 FP16 A100/A800 3090 INT4/8 3090

MOSS

DB-GPT

DB-GPT GPT GPT 100% 100%

DB-GPT

CPM-Bee

CPM-Bee Transformer auto-regressive

CPM-Bee

• OpenBMB CPM-Bee
• CPM-Bee
• CPM-Bee
• OpenBMB OpenBMB CPM-Bee
• OpenBMB CPM-Bee

CPM-Bee

LaWGPT

LaWGPT

Chinese-LLaMAChatGLM

(Linly)

1. 32*A100 GPU baseline33B Linly-Chinese-LLAMA LLaMA
3. CUDA CPU Huggingface

• Linly-Chinese-LLaMA LLaMA 7B13B 33B 65B
• Linly-ChatFlow 400 7B13B
• Linly-ChatFlow-int4 ChatFlow 4-bit CPU

• Linly-Chinese-BLOOM BLOOM 7B 175B

Chinese-Vicuna LLaMA

Chinese-Vicuna LLaMA+Lora

• finetune
• CPU ( C++)
• / / Facebook llama.ckpt

Chinese-LLaMA-Alpaca LLaMA & Alpaca

Chinese-LLaMA-Alpaca LLaMA Alpaca

LLaMA Alpaca

ChatYuan

ChatYuan ChatYuan-large-v2 v1

ChatYuan-large-v2 ChatYuan PC INT4 400M

GPT

HuatuoGPT GPT ChatGPT

HuatuoGPT ChatGPT

LLaMA

(BenTsao) (HuaTuo) LLaMA

/ (Instruct-tuning) LLaMA-7B GPT3.5 API LLaMA LLaMA

2000 NPUGPU

• common crawl 80TB 1.1TB 53 2TB
• NLP

(PanGu-Dialog)

PanGu-Dialog

• () PGCED12
• +
• SOTA

1.75 7

• CogView
• CogView 40 MS COCO OpenAI DALLE
• BriVL
• BriVL (Bridging Vision and Language Model) BriVL UNITERCLIP

• GLM
• GLM BERTRoBERTa T5 1.1 3.35 4.10 5.15 100
• CPM
• CPM 26 110 1980
• Transformer-XL
• Transformer-XL 29 / NLG
• EVA
• EVA 28 14 WDC
• Lawformer
• Lawformer 1

• ProtTrans
• ProtTrans 30

BBT-2 120

BBT-2 120 BBT-2 BBT-2

• BBT-2-12B-Text120
• BBT-2.5-13B-Text: 130 +
• BBT-2-12B-TC-001-SFT
• BBT-2-12B-TF-001 120
• BBT-2-12B-Fig
• BBT-2-12B-Science

BELLE

BELLE: Be Everyone's Large Language model Engine

LLM Engine BLOOM ChatGPT

TigerBot

TigerBot (LLM) OpenAI InstructGPT NLP TigerBot-7B OpenAI 96%

YuLan-Chat

YuLan-Chat

LLM

LLaMA Meta

LLaMA "Large Language Model Meta AI" Meta LLM 70 130 330 650

LaMA-13B130 OpenAI GPT-31750 GPT-3 PC ChatGPT AI

Stanford Alpaca LLaMA

Stanford Alpaca Alpaca LLaMA Meta LLaMA 7B

Stanford Alpaca OpenAI text-davinci-003 self-instruct 52K instruction-following Alpaca

Lit-LLaMA nanoGPT

Lit-LLaMA nanoGPT LLaMA LoRA flash attentionLLaMA-Adapter Int8 GPTQ 4bit

Lit-LLaMA LLaMA GPL GPL Lit-LLaMA

GloVe

GloVeGlobal Vectors for Word Representationcount-based & overall statisticsword representationsimilarityanalogycosine

GloVe Public Domain Dedication and License

• Wikipedia 2014+Gigaword 5(6B tokens, 400K vocab, uncased, 50d, 100d, 200d, & 300d vectors, 822 MB download):glove.6B.zip
• Common Crawl (42B tokens, 1.9M vocab, uncased, 300d vectors, 1.75 GB download):glove.42B.300d.zip
• Common Crawl (840B tokens, 2.2M vocab, cased, 300d vectors, 2.03 GB download):glove.840B.300d.zip
• Twitter (2B tweets, 27B tokens, 1.2M vocab, uncased, 25d, 50d, 100d, & 200d vectors, 1.42 GB download):glove.twitter.27B.zip

Dolly

Dolly LLMDolly EleutherAI 60

60 ChatGPT 1750 Dolly ChatGPT ""

OPT-175B Meta

OPT-175B Meta 1750 GPT-3 GPT-3OPT-175B

Meta OPT-175B OPT-175B Meta OPT-175B OPT-175B

Cerebras-GPT

Cerebras GPT Cerebras 1.11 130 7

Cerebras-GPT

BLOOM

Bloom 1760 46 13 GitHub Copilot

BLOOM Hugging Face 1760 BLOOM

BLOOMChat 176B LLM

BLOOMChat LLMSambaNova Together SambaNova SambaNova DataScale BLOOMChat BigScience BLOOM OpenChatKitDolly 2.0 OASST1 OIG

GPT-J AI

GPT-J GPT-3 60 AI

800GB GPT-3 Google Cloud v3-256 TPU EleutherAI The Pile GPT-J NLP OpenAI 67 GPT-3 Colab EleutherAI

GPT-2 Transformer

GPT-2 transformer 15 800

GPT-2

GPT-2 OpenAI 2018 GPT

RWKV-LM Transformer

RWKV RNN Transformer

RWKV Transformer Block Transformer Block RWKV self-attention Position Encoding TimeMix FFN ChannelMix Transfomer

LoRA

LoRA LLaMA LLaMA GPU

-7B13B 30B - /

Demo

CodeGeeX

CodeGeeX 130 CodeGeeX MindSpore II 192 1536 910 AI

CodeGeeX

• PythonC++JavaJavaScript Go HumanEval-X 47%~60%
• HumanEval-X
• CodeGeeX VSCode
• : CodeGeeX 910 V100/A100

Falcon LLM

FalconTIIFalconLLaMATIIFalconFalcon 40B400650LLaMA

Vicuna LLaMA

Vicuna LLaMA MBZUAI 7B 13B

Vicuna-13B Stanford Alpaca

GPT-4 Vicuna-13B OpenAI ChatGPT Google Bard 90% 90% LLaMA Stanford Alpaca Vicuna-13B 300

RedPajama 1.2

RedPajama LLaMA 1.2 token TogetherOntocord.aiETH DS3Lab CRFMHazy Research MILA AI

RedPajama

OpenAssistant

OpenAssistant AI

OpenAssistant Conversations 60 LLaMA 13B 30B

StableLM Stability AI

StableLM Stability AI StableLM Stability AI StableLM-alpha 30 70 150 300

StableLM

StarCoder AI

StarCoder150 Hugging Face ServiceNow GitHub Copilot CodeWhisperer AI

SantaCoder AI

SantaCoder 11 PythonJava JavaScript

SantaCoder The Stackv1.1SantaCoder 11 InCoder67 CodeGen-multi27 SantaCoder

MLC LLM

MLC LLM

MLC LLM MLC LLM AI GPU

Web LLM

Web LLM LLM Web WebGPU AI GPU

WizardLM LLaMA

WizardLM 7B LLaMA LLM

WizardLM Evol-Instruct LLM LLM 70k

YaLM 100B

YaLM 100B GPT

1000 800 A100 1.7 TB 65

OpenLLaMA LLaMA

OpenLLaMA Meta AI LLaMA

2000 7B OpenLLaMA OpenLLaMA PyTorch Jax LLaMA

LLM

LangChain LLM

LangChain LLM LLM

LangChain

• Prompts
• LLMs LLM LLM
• Document Loaders
• Utils Python REPLLangChain
• ChainsChains LLM LLM LangChain
• Indexes -
• AgentsAgents LLM LangChain
• MemoryMemory / LangChain /
• ChatChat API - LangChain

JARVIS LLM AI

JARVIS LLM AI LLMAI HuggingFace Hub

• ChatGPT
• ChatGPT Hugging Face AI
• ChatGPT
• : ChatGPT Response

Semantic Kernel LLM SDK

Semantic Kernel SDK AI (LLM)

Semantic Kernel AI

Semantic Kernel AI /

LMFlow

LMFlow

LMFlow

LMFlow

LLaMA GPT-2Galactica

xturing LLM

xturing LLM LLaMAGPT-JGPT-2OPTCerebras-GPTGalactica LLMxTuring LLM

xturing

• LLM
• GPU GPU
• LoRA 90%

Dify LLMOps

Dify LLMOps AI Dify API

Dify Define Modify AI Dify Do it for you

Dify

• Web
• API

Dify Langchain LLMs

• GPT 3 (text-davinci-003)
• GPT 3.5 Turbo(ChatGPT)
• GPT-4

Dify.AI** **

• Prompt prompt AI
• API API
• AI AI

• Notion Notion AI
• ChatGPT Dify ChatGPT Dify
• Llama Llama

Flowise LLM

Flowise UI Node Typescript/Javascript LangchainJS LLM

• LLM Chain LLM LLM Chain

• Language Translation Chain LLM Chain

Jigsaw Datase

Jigsaw GPT-3Codex

Jigsaw Python Pandas API Pandas API manipulating dataframes

Codex

Jigsaw Jigsaw GPT-3 Codex

Jigsaw 30%

GPTCache LLM

GPTCache LLM LLM API 10 100

ChatGPT LLM LLM API LLM GPTCache LLM

LLM

chatGLM-6BchatRWKVchatYuan chatGLM-6B

1. chatGLM-6BchatRWKVchatYuan
4. chatGLM-6BchatRWKV

MindFormers //

MindSpore MindFormers Transformer SOTA

MindSpore MindFormers MindSpore

• TrainerModelClassConfigClasspipeline

• BERT
• GPT
• OPT
• T5
• MAE
• SimMIM
• CLIP
• FILIP
• Vit
• Swin

Code as Policies

Code as Policies CaP PaLM-SayCan Python CaPGoogle CaP CaP

Colossal-AI

ColossalAI

Colossal-AI

ColossalAI GPU

Disclaimer: The content of this article is sourced from the internet. The copyright of the text, images, and other materials belongs to the original author. The platform reprints the materials for the purpose of conveying more information. The content of the article is for reference and learning only, and should not be used for commercial purposes. If it infringes on your legitimate rights and interests, please contact us promptly and we will handle it as soon as possible! We respect copyright and are committed to protecting it. Thank you for sharing.(Email:[email protected])