Telegram Group & Telegram Channel
πŸ–₯ Phi-3-mini Π² 30 строках Π½Π° C# с ONNX Runtime GenAI

Π’ Ρ€Π°ΠΌΠΊΠ°Ρ… запуска Phi-3 Microsoft выпустила ΠΎΠΏΡ‚ΠΈΠΌΠΈΠ·ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹Π΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ONNX, ΠΊΠ°ΠΊ ΠΏΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎ описано Π² ΡΡ‚Π°Ρ‚ΡŒΠ΅ Β«ONNX Runtime supports Phi-3 mini models across platforms and devicesΒ».
Π’Π°ΠΊΠΆΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ Phi-3 mini ΠΎΠΏΡƒΠ±Π»ΠΈΠΊΠΎΠ²Π°Π½Ρ‹ Π½Π° HuggingFace πŸ€—

Π˜ΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ всё это, ΠΌΠΎΠΆΠ½ΠΎ Π»Π΅Π³ΠΊΠΎ Π·Π°ΠΏΡƒΡΡ‚ΠΈΡ‚ΡŒ модСль локально всСго Π² Π½Π΅ΡΠΊΠΎΠ»ΡŒΠΊΠΈΡ… строках C#, ΠΊΠ°ΠΊ ΠΏΠΎΠΊΠ°Π·Π°Π½ΠΎ Π² этом Π³Π°ΠΉΠ΄Π΅.

▢️ Π“Π°ΠΉΠ΄

А Π²ΠΎΡ‚ Ρ‚Π΅ самыС 30 строк:

using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
@"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
Console.Write("Prompt: ");
var line = Console.ReadLine();
if (line == null) { continue; }

using var tokens = tokenizer.Encode(line);

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetInputSequences(tokens);

using var generator = new Generator(model, generatorParams);

while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}


@csharp_1001_notes
Please open Telegram to view this post
VIEW IN TELEGRAM



tg-me.com/csharp_1001_notes/456
Create:
Last Update:

πŸ–₯ Phi-3-mini Π² 30 строках Π½Π° C# с ONNX Runtime GenAI

Π’ Ρ€Π°ΠΌΠΊΠ°Ρ… запуска Phi-3 Microsoft выпустила ΠΎΠΏΡ‚ΠΈΠΌΠΈΠ·ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹Π΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ONNX, ΠΊΠ°ΠΊ ΠΏΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎ описано Π² ΡΡ‚Π°Ρ‚ΡŒΠ΅ Β«ONNX Runtime supports Phi-3 mini models across platforms and devicesΒ».
Π’Π°ΠΊΠΆΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ Phi-3 mini ΠΎΠΏΡƒΠ±Π»ΠΈΠΊΠΎΠ²Π°Π½Ρ‹ Π½Π° HuggingFace πŸ€—

Π˜ΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ всё это, ΠΌΠΎΠΆΠ½ΠΎ Π»Π΅Π³ΠΊΠΎ Π·Π°ΠΏΡƒΡΡ‚ΠΈΡ‚ΡŒ модСль локально всСго Π² Π½Π΅ΡΠΊΠΎΠ»ΡŒΠΊΠΈΡ… строках C#, ΠΊΠ°ΠΊ ΠΏΠΎΠΊΠ°Π·Π°Π½ΠΎ Π² этом Π³Π°ΠΉΠ΄Π΅.

▢️ Π“Π°ΠΉΠ΄

А Π²ΠΎΡ‚ Ρ‚Π΅ самыС 30 строк:


using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
@"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
Console.Write("Prompt: ");
var line = Console.ReadLine();
if (line == null) { continue; }

using var tokens = tokenizer.Encode(line);

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetInputSequences(tokens);

using var generator = new Generator(model, generatorParams);

while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}


@csharp_1001_notes

BY C# 1001 notes





Share with your friend now:
tg-me.com/csharp_1001_notes/456

View MORE
Open in Telegram


C 1001 notes Telegram | DID YOU KNOW?

Date: |

Mr. Durov launched Telegram in late 2013 with his brother, Nikolai, just months before he was pushed out of VK, the Russian social-media platform he founded. Mr. Durov pitched his new appβ€”funded with the proceeds from the VK saleβ€”less as a business than as a way for people to send messages while avoiding government surveillance and censorship.

Why Telegram?

Telegram has no known backdoors and, even though it is come in for criticism for using proprietary encryption methods instead of open-source ones, those have yet to be compromised. While no messaging app can guarantee a 100% impermeable defense against determined attackers, Telegram is vulnerabilities are few and either theoretical or based on spoof files fooling users into actively enabling an attack.

C 1001 notes from fr


Telegram C# 1001 notes
FROM USA