Speculative classification

In this example, we will use the classification speculation preset to determine the sentiment of the user’s input.

Python
Swift
TypeScript
Rust

Create a new project

uv init demo && cd demo

Install dependencies

uv add uzu

Paste into main.py

Run the snippet

uv run main.py

Create a new SwiftUI project

Add the SDK

Add this package through SPM:

https://github.com/trymirai/uzu.git

Paste the snippet

Add the snippet call

var body: some View {
    VStack {
        Text("On-device AI")
    }
    .onAppear() {
        Task {
            try await runExampleName()
        }
    }
}

Make a new project folder

mkdir demo && cd demo

Initialize with pnpm

pnpm init

Install dependencies

pnpm add typescript ts-node @types/node -D
pnpm add @trymirai/uzu

Initialize a tsconfig.json

{
    "compilerOptions": {
        "target": "es2020",
        "module": "commonjs",
        "moduleResolution": "node",
        "strict": true,
        "esModuleInterop": true,
        "outDir": "dist",
        "types": [
            "node"
        ]
    },
    "include": [
        "*.ts"
    ]
}

Create main.ts

Run the snippet

pnpm ts-node main.ts

Create a new project

cargo new demo && cd demo

Install dependencies

cargo add uzu --git https://github.com/trymirai/uzu
cargo add tokio --features full

Paste into src/main.rs

Run the snippet

cargo run --release

You can view the stats to see that the answer will be ready immediately after the prefill step, and actual generation won’t even start due to speculative decoding, which significantly improves generation speed.

Overview

Quick start

Speculative classification