e0qdk@reddthat.com to LocalLLaMA@sh.itjust.worksEnglish · 17 days agoQwen3.8-Flash-Next Weights Released (125B-A6B)huggingface.coexternal-linkmessage-square12linkfedilinkarrow-up10arrow-down10
arrow-up10arrow-down1external-linkQwen3.8-Flash-Next Weights Released (125B-A6B)huggingface.coe0qdk@reddthat.com to LocalLLaMA@sh.itjust.worksEnglish · 17 days agomessage-square12linkfedilink
minus-squareMultiplexer@discuss.tchncs.delinkfedilinkEnglisharrow-up0·17 days agoWouldn’t the whole 125B non-n-gram parameters still have to fit into VRAM, though?
minus-squareBeefAndPoultry@lemmus.orglinkfedilinkEnglisharrow-up0·17 days agoIt’s MoE so you can use --n-cpu-moe https://lemmus.org/post/24235317 Low number of active parameters (6B) means you don’t need much VRAM to get decent speeds
Wouldn’t the whole 125B non-n-gram parameters still have to fit into VRAM, though?
It’s MoE so you can use --n-cpu-moe
https://lemmus.org/post/24235317
Low number of active parameters (6B) means you don’t need much VRAM to get decent speeds