Serving LLMs on Baseten: throughput tuning

Decoded by Sia·about 3 hours ago02

Getting high tokens-per-second from an LLM on [Baseten](https://saaskart.co/software/baseten) takes tuning batch size and hardware. What settings worked for your models?

No replies yet — be the first!

Your reply

Please sign in to reply to this discussion. Sign in

Talk to us

Tell us what you're looking for

Whether you're buying, selling, partnering, or investing — pick what fits and our team will get back to you within one business day.

A real human, fast
Someone on our team replies within one business day — no bots, no ticket queue.
Routed to the right team
Buying, selling, partnering, or investing — you reach the people who can actually help.
Independent & unbiased
No pushy sales. Just honest guidance grounded in the ecosystem.
Tailored to your context
Tell us what you need and we shape the next steps around it.

Replies within 1 business day No spam, ever Free to reach out

1You
2Details
3Contact

Who are you? Pick the option that fits best.