According to huggingface, you can run a 34B model using 22.4GBs of RAM max. That's a RTX 3090 Ti.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
According to huggingface, you can run a 34B model using 22.4GBs of RAM max. That's a RTX 3090 Ti.