The memory is plausible, but the FP8 / INT8 + offload profile needs an OpenMayhem benchmark first.
- Model size
- 20B · 50-step reference
- Model format
- FP8 / INT8 + offload
- Runs with
- vLLM-Omni
This profile is provisional and needs a local OpenMayhem benchmark before serving.