conv.

← The whole story
AI
Two arXiv papers examine efficiency of speculative decoding techniques

Hasan publishes inference engineering explainer on speculative decoding

2 Sep 15 4:45 PM · 12d ago · 1 post · 1 source · development 2 of 2

Engineer Danial Hasan publishes an article explaining speculative decoding as part of an inference engineering series, detailing how the technique speeds LLM inference and when it saves time, with practical examples using open-source vLLM.

Hasan publishes inference engineering explainer on speculative decoding
x.com

“Speculative decoding can reduce the time an AI assistant takes to finish an answer. A cheap proposer drafts several tokens, and the model serving the request checks them together.”

Danial Hasan

Jahyun Koo, Sunghyeon Woo, Jaeeun Kil, Jeongtae Lee, Sungjae Lee, Kyomin Jung, Minsub Kim ResearchersIlya Koziev, Leonid Sinev, Ivan Oseledets ResearchersDanial Hasan Inference engineer

The whole story articlesposts the bright band is this development · numbered dots are the others · click one to jump

Peak 2 pieces in 3h at Sep 14, 11 PM; 3 pieces over 13 days (2 articles · 1 post) Sep 14, 11 PM — 2 pieces · 2 articles — Newswires 2Sep 15, 2 AM — quietSep 15, 5 AM — quietSep 15, 8 AM — quietSep 15, 11 AM — quietSep 15, 2 PM — 1 piece · 1 post — X 1Sep 15, 5 PM — quietSep 15, 8 PM — quietSep 15, 11 PM — quietSep 16, 2 AM — quietSep 16, 5 AM — quietSep 16, 8 AM — quietSep 16, 11 AM — quietSep 16, 2 PM — quietSep 16, 5 PM — quietSep 16, 8 PM — quietSep 16, 11 PM — quietSep 17, 2 AM — quietSep 17, 5 AM — quietSep 17, 8 AM — quietSep 17, 11 AM — quietSep 17, 2 PM — quietSep 17, 5 PM — quietSep 17, 8 PM — quietSep 17, 11 PM — quietSep 18, 2 AM — quietSep 18, 5 AM — quietSep 18, 8 AM — quietSep 18, 11 AM — quietSep 18, 2 PM — quietSep 18, 5 PM — quietSep 18, 8 PM — quietSep 18, 11 PM — quietSep 19, 2 AM — quietSep 19, 5 AM — quietSep 19, 8 AM — quietSep 19, 11 AM — quietSep 19, 2 PM — quietSep 19, 5 PM — quietSep 19, 8 PM — quietSep 19, 11 PM — quietSep 20, 2 AM — quietSep 20, 5 AM — quietSep 20, 8 AM — quietSep 20, 11 AM — quietSep 20, 2 PM — quietSep 20, 5 PM — quietSep 20, 8 PM — quietSep 20, 11 PM — quietSep 21, 2 AM — quietSep 21, 5 AM — quietSep 21, 8 AM — quietSep 21, 11 AM — quietSep 21, 2 PM — quietSep 21, 5 PM — quietSep 21, 8 PM — quietSep 21, 11 PM — quietSep 22, 2 AM — quietSep 22, 5 AM — quietSep 22, 8 AM — quietSep 22, 11 AM — quietSep 22, 2 PM — quietSep 22, 5 PM — quietSep 22, 8 PM — quietSep 22, 11 PM — quietSep 23, 2 AM — quietSep 23, 5 AM — quietSep 23, 8 AM — quietSep 23, 11 AM — quietSep 23, 2 PM — quietSep 23, 5 PM — quietSep 23, 8 PM — quietSep 23, 11 PM — quietSep 24, 2 AM — quietSep 24, 5 AM — quietSep 24, 8 AM — quietSep 24, 11 AM — quietSep 24, 2 PM — quietSep 24, 5 PM — quietSep 24, 8 PM — quietSep 24, 11 PM — quietSep 25, 2 AM — quietSep 25, 5 AM — quietSep 25, 8 AM — quietSep 25, 11 AM — quietSep 25, 2 PM — quietSep 25, 5 PM — quietSep 25, 8 PM — quietSep 25, 11 PM — quietYesterday, 2 AM — quietYesterday, 5 AM — quietYesterday, 8 AM — quietYesterday, 11 AM — quietYesterday, 2 PM — quietYesterday, 5 PM — quietYesterday, 8 PM — quietYesterday, 11 PM — quietToday, 2 AM — quietToday, 5 AM — quietToday, 8 AM — quietToday, 11 AM — quietToday, 2 PM — quietToday, 5 PM — quietToday, 8 PM — quiet 12
1Koziev et al. question Orthrus's lossless…2Hasan publishes inference engineering…
Sep 15Sep 16Sep 17Sep 18Sep 19Sep 20Sep 21Sep 22Sep 23Sep 24Sep 25yesterdaynow · 11:44 PM ET

What people said 1 voice · verbatim

All 2 developments of Two arXiv papers examine efficiency of speculative decoding… →

NewswiresX