K2 Horizon 0.9B Model Beats Llama 3.2 1B by 25%: Here’s How We Did It

➤ • 8GB RAM Test Device

➤ Model: IFM/K2-Horizon-0.9B (4-bit Q4_K_M GGUF via llama.cpp)

---

A journey through on-device model deployment: how we benchmarked K2 Horizon against Llama 3.2 across 500 test cases, tackled mobile heating and battery drain on an 8GB Android device, and achieved a +25% performance edge by turning our SLM into an Ahead-of-Time Rule Compiler.

We originally set out to test the new K2 Horizon model family locally. We first benchmarked both the 0.9B and 7B models on a 32GB RAM laptop by building a local chatbot testbed. While the 7B model delivered solid answers, its latency and memory load were noticeably heavier compared to the nimble 0.9B variant. For mobile deployment on Android, the 0.9B model was the clear contender. Given K2's strong reasoning capabilities even at smaller parameter sizes, we wanted to build a practical real-world use case, which led us to create an on-device Notification Analyzer.

1. The Everyday Problem: The Important Notification You Missed Because You Were Busy

Every day, our phones get flooded with notification noise. When busy at work, in meetings, or sleeping at night, muting notifications is the easiest fix, but it comes at a cost: you risk missing the alerts that actually matter.

We built an on-device Android notification analyzer where you define priority rules in plain English. When an alert matches your rules, the app alerts you through a high-priority media sound chime and custom vibration, even if your phone's notification volume is muted or set to zero. It is 100% private because everything runs locally using K2 Horizon 0.9B, a compact open-weights Small Language Model (SLM).

2. Development & Architecture: Tackling Heat with Ahead-of-Time Compilation

Deploying a language model directly on a mobile device introduces strict thermal and power constraints. Unlike a server with dedicated cooling, mobile processors quickly throttle under continuous load. Here is how our early experiments unfolded:

● Attempt 1 — BF16 0.9B (~2.16 GB), inference on every notification: Loaded fine into RAM, but heavy continuous CPU compute on each incoming alert caused immediate phone heating and rapid battery drain.

● Attempt 2 — RAM residency & lazy loading (permanent vs. idle timeout): Continuous RAM residency created constant memory pressure, while lazy unloading triggered repeated cold-starts and CPU spikes during notification bursts.

Running model inference on every incoming notification was fundamentally the wrong design for an always-on background service. That led to rethinking the entire architecture from first principles:

"For simple rules, why should an SLM run at runtime to decide if an alert is important? We can delegate easy tasks to a fast classifier."

Instead of treating K2 Horizon as a runtime filter that evaluates every message, we convert it into an Ahead-of-Time (AOT) Rule Compiler. The language model primarily runs when you create or edit a rule, translating plain-English intent into deterministic matching logic, with on-demand inference for rules that require semantic reasoning.

Here is how the dual-engine pipeline operates in practice:

● 1. At Rule Creation Time (AOT Compilation): When you type a rule like "if any msg from madhu it is important, if she sends reels it is not important", K2 processes it once. It extracts the contact name, required topic triggers, and excluded keywords into a structured JSON schema.

AOT compilation still has a compute cost. Because K2 performs inference during rule creation or editing, creating or editing multiple rules consecutively can produce temporary CPU activity and slight device heating. This cost is paid during configuration rather than continuously on every incoming notification.

● 2. Fast Classifier Runtime Path (<0.2 ms): Once rules are compiled, simple notifications are handled by the deterministic fast path without repeatedly invoking the model. When a notification arrives, the on-device classifier evaluates it against the compiled schema. App filters, contact lookups, and keyword exclusions are processed in less than 0.2 milliseconds with zero CPU spikes and zero thermal buildup.

● 3. Complex Semantic Rules (K2 On-Demand): If a rule explicitly requires sentiment analysis or emotional tone (e.g. "any message from Pranav when he is angry it is not important"), the fast classifier routes only those specific matching messages to K2 Horizon for deep semantic reasoning.

3. Stress Testing & Benchmarks: K2 Horizon 0.9B vs. Llama 3.2 1B

Before deploying to daily-driver testing, we conducted rigorous stress tests across a benchmark dataset of 500 diverse test cases. The test suite covered complex user emotional tones (sarcasm, passive aggressive frustration, anger), urgent emergency escalation, conditional exclusions, and app-specific rules. The reported accuracy reflects end-to-end performance of our notification-analysis pipeline using each model’s compiled rules, deterministic matching, and semantic escalation where applicable.

We benchmarked K2 Horizon 0.9B against Llama 3.2 1B using the same 500-case dataset and evaluation harness. K2 performed better than Llama 3.2 1B on our notification-analysis benchmark:

Key Benchmark Metrics (500 Test Cases):

● Overall Triage Accuracy: K2 Horizon 0.9B scored 66.2% (331/500) vs. Llama 3.2 1B at 41.0% (205/500) (+25.2% Higher Overall Accuracy).

● AOT Rule Compilation (15 Natural Rules): K2 Horizon 0.9B achieved 86.7% (13/15) vs. Llama 3.2 1B at 26.7% (4/15).

● Deep AI Emotional Nuance (250 cases): K2 Horizon 0.9B scored 58.4% vs. Llama 3.2 1B at 48.4%.

These results come from our internal notification-analysis benchmark and should not be interpreted as a general ranking of K2 Horizon 0.9B versus Llama 3.2 1B across language-model tasks. In accordance with system semantics, suppression actions (IGNORE ≡ MUTE) are normalized during evaluation.

● K2 Horizon 0.9B Strengths: Exceptional structured JSON output generation, high accuracy in parsing complex multi-clause rules (86.7%), and reliable passive aggressive sentiment detection.

● Llama 3.2 1B Bottlenecks: Struggled to reliably output valid JSON schemas for multi-condition app rules, resulting in frequent rule compilation failures (26.7% accuracy).

Sample Stress Test Cases:

Stress Case 1: Passive-Aggressive Frustration Detection

Stress Case 2: Critical Production Escalation

Case Breakdown: While both models reliably catch explicit emergency keywords like "Urgent" (right), K2 Horizon 0.9B successfully identifies nuanced conversational cues like passive aggressive frustration (left) to silence unneeded alerts, where Llama 3.2 1B misclassifies dismissiveness as neutral agreement.

Between K2 Horizon 0.9B and Llama 3.2 1B, K2 proved to be the better fit for our notification-analysis workload. As a dedicated reasoning model, its reasoning-oriented design and strict instruction following allow it to reliably compile complex natural language rules into structured schemas and decipher subtle human emotions where Llama 3.2 1B falls short.

4. Real-World Testing on Q4 Version of K2-0.9B

Deploying with 4-Bit Quantization (Q4_K_M)
 
For everyday mobile deployment on an Android device, a 4-bit quantized 
version of K2 Horizon (Q4_K_M, ~666 MB) provided a practical balance of 
model size, latency, and on-device performance in our testing. It reduces 
model artifact size by ~69% (from ~2.16 GB to ~666 MB), lowers 
inference latency, and showed no noticeable thermal buildup during
our 5-day daily-driver testing.

Below are screens from testing the application in daily use on Android:

Active Rules Screen

Rules categorized by K2 Horizon into Fast Classifier vs. Deep AI tiers

HOW K2 CATEGORIZED EACH RULE:

● "any message from pranav when he is angry it is not important" ➔ K2 Deep AI (On-Demand)

● "anyone msges about playing cricket it is important" ➔ AOT Topic Filter (<0.2ms)

● "if any msg from madhu it is important,if she sends reels it is not important" ➔ AOT Conditional (<0.2ms)

● "any msg from arjun related to job is important" ➔ AOT Conditional (<0.2ms)

● "any message from teams is important" ➔ App Filter (<0.2ms)

● "any msg from krishna it is important" ➔ Fast Contact (<0.2ms)

Live Notification Feed Testing

Real-time alert triage, conditional topic matching, and quiet suppression

HOW EACH NOTIFICATION WAS TRIAGED BASED ON THE ABOVE RULES:

● Madhu: "Sent you a reel" ➔ Silenced (Reel Excluded)

● Madhu: "Hi" ➔ Alerted (Important Contact)

● Arjun: "I found a job opening on LinkedIn" ➔ Alerted (Topic: 'job')

● Arjun: "Lets go to a trip" / "Watched movie?" ➔ Silenced (No Match)

5-Day Daily-Driver Testing

● Test Duration: 5 continuous days as a daily driver (7 active rules).

● Notification Volume: 500 notifications/day (2,500 total processed across WhatsApp, Teams, Instagram, Phone, etc.).

● Observations: We observed only 2–3 minor edge cases across approximately 2,500 notifications.

● Efficiency: No measurable additional standby battery drain and no noticeable thermal buildup during our 5-day test.

5. Key Takeaways

By pairing K2 Horizon with a fast on-device classifier, we use the model where it excels: compiling complex human intent into structured rules ahead of time, while native code handles instant runtime execution. This architecture delivers deep semantic flexibility without sacrificing the battery life or thermal stability of the host device.

Small language models do not need to process every single incoming data stream at runtime. When building on-device AI applications, the winning formula is not making the model compute faster on every event, but using compact SLMs as intelligent compilers and letting deterministic code handle the high-throughput path.