Advice for 5.000 € Workstation
I have been tasked to buy a 5.000 € inference machine.
Goals
- Inference for Llama 3.1 70b (in 4Q) -> so likely 48 GB of VRAM are needed
- Finetuning (if possible) for smaller 8b models
- Should run indefinetly as a linux inference server (dev only) in our office
Can anyone recommend me any pre-built solutions or specific specs for such a workstation?
Cheers!