Mingmin originates from Aofei TempleQuantum official account | QbitAIMojo, a new language specifically designed for AI, infers LLaMA2 performance250 times better than Python!Even faster than C language20%.Mojo

Mingmin originates from Aofei Temple

Quantum official account | QbitAI

Mojo, a new language specifically designed for AI, infers LLaMA2 performance250 times better than Python!

Even faster than C language20%.

Mojo.

MojoPython6.8.

.

Mojo.

OpenAIKarpathy.

Currently, LLaMA.Mojo has been opened for download~

Download in just a few lines of code

The guy who brought this version is a former Meta engineerAydynTairov.

He used Mojo'sSIMD(SingleInstructionMultipleData) andVectorized primitiveTransforming llama2. py into Mojo has improved performance by nearly250x.

Even in fast running mode, the Mojo version performs better than the C language version15-20%.

Mojo.

The system and hardware comparison conducted by the author is as follows:

Mojo.

First, save the repository to the folder where the project was saved:

GitCloneHttps://Github. com/tairov/lama2. Mojo. git

Then open the storage folder:

CDLlama2. Mojo

You can download the model below:

WgetHttps://Huggingface.co/karpati/Tinyllamas/resolve/Main/categories15M. bin

Then you can run:

Mojo llama2.MojoNum Hardwarethreads: 6SIMDvectorwidth: 8Checkpoint Size: 60816028≪ S>Once Uponatime, therewasalittlegirlnamedLily. Shelvedtoplayoutside in the sun Oneday, shesawabig, redballinthesky. ITlasthesun! ShethoughtitwassoprettyLily Wantedtoplay with the ball, but it was too high up in the sky. ShetriedtoJumpandreachit, but she could not. Then, shehadanidea. Shehoulduseaseaseaseaseaseaseaseaseaseaseasea lockdown of the ball Lily found a stick and tried to hit the ball Butthestickwastoshort.Shetriedagainandagain, but she could not treat it. ShefeltsadSuddenly, AKINDMANCAMEBY and SAWLily. Headedherwhatwaswrong. Lilytoldhimabouttheball. Themansmiledandsay, "I have ausefulide!" He tookoutalongstick and usedittoknocktheballdown Lilywassohappy! Shethankened the manager and they lay yedtogether in the sunshine≪ S>Once Uponatime, therewasalittlegirlnamedLily. Shelvedtoplayoutside in the sun Oneday, shesawabig, redAchieved Tok/s: 264.24870466321244

Why is Mojo so fast?

But then again, why can Mojo's speed be so fast?

Mojo.

It was born in May this year,Developed specifically for the AI field, byFather of LLVMandFather of SwiftChris Lattner.

PythonandC++Simple syntax and fast operationandPython.

Since its launch, Mojo has attracted120000 developersGitHub9K.

8MojoModular11.3.

Mojo4.

Step 1, bytype annotation Eliminate the loss of Python dynamic types and doAlgebraic simplification(Algebraic simplifications), avoiding square root operations and simplifying complex square operations to achieve89x.

Step 2, implement through vectorizationSIMDParallel computation of (single instruction multiple data), with vector width matching the number of FMA (floating-point multiplication and accumulation units) in the CPU, achieving874x.

Step 3, change the single threaded implementation developed in the first two steps toMulticore parallelizationFor an 88 core system, an additional 30 times acceleration has been achieved compared to the original Python26000x.

Step 4, solve the problem of parallelismLoad imbalance issueEnable threads to dynamically obtain tasks from the pool and obtain the final result68000x.

Mojo.LinuxMacandWindows.

VSCodeand.

PythonJupyter.

Interested children's shoes can be experienced firsthand~

GitHub address:
Https://github.com/tairov/llama2.Mojo

Mojo Document:
Https://docs.modular.com/Mojo/manual/get-started/index.html

- End -

Follow us and stay informed of cutting-edge technology trends as soon as possible

Disclaimer: The content of this article is sourced from the internet. The copyright of the text, images, and other materials belongs to the original author. The platform reprints the materials for the purpose of conveying more information. The content of the article is for reference and learning only, and should not be used for commercial purposes. If it infringes on your legitimate rights and interests, please contact us promptly and we will handle it as soon as possible! We respect copyright and are committed to protecting it. Thank you for sharing.(Email:[email protected])