Phi-4-mini-instruct-w4a4-fp4

This is a quantized version of the Phi-4-mini-instruct model by Microsoft, quantized using NVFP4 format with 4-bit weights and 4-bit activations.

Model Description

Phi-4-mini-instruct is a smaller variant of the Phi-4 model, designed for instruction-following tasks. The quantized version retains much of the original model's capabilities while significantly reducing its size and computational requirements.

Quantization Details

• Quantization Format: NVFP4
• Weight Precision: 4-bit
• Activation Precision: 4-bit
• Quantization Tool: LLM Compressor