Large Language Models

35% REAP Quantization Runs 397B Model on 96GB GPU

A community researcher achieved usable quality from a 397B parameter model using 35% REAP quantization on a 96GB GPU.