▲ 66 ▼ Mac Studio With M3 Ultra Runs Massive DeepSeek R1 AI Model Locally (www.macrumors.com) submitted 1 year ago by cantankerous_cashew@lemmy.world to c/apple_enthusiast@lemmy.world 5 comments fedilink hide all child comments
[–] vanderbilt@lemmy.world 3 points 1 year ago (1 child) Unfortunately getting an AI workload to run on those XTXs, and run correctly, is another story entirely. permalink fedilink source parent hideshow 2 child comments replies: [–] timewarp@lemmy.world 3 points 1 year ago* (1 child) ROCm has made a lot of improvements. $2000 for 48GB of VRAM makes up for any minor performance decrease as opposed to spending $2200 or more for 24GB VRAM with NVIDIA. permalink fedilink source parent hideshow 2 child comments replies: [–] vanderbilt@lemmy.world 1 point 1 year ago ROCm certainly has gotten better, but the weird edge cases remain; alongside the fact that merely getting certain models to run is problematic. I am hoping that RNDA4 is paired with some tooling improvements. No more massive custom container builds, no more versioning nightmares. At my last startup we tried very hard to get AMD GPUs to work, but there were too many issues. permalink fedilink source parent
[–] timewarp@lemmy.world 3 points 1 year ago* (1 child) ROCm has made a lot of improvements. $2000 for 48GB of VRAM makes up for any minor performance decrease as opposed to spending $2200 or more for 24GB VRAM with NVIDIA. permalink fedilink source parent hideshow 2 child comments replies: [–] vanderbilt@lemmy.world 1 point 1 year ago ROCm certainly has gotten better, but the weird edge cases remain; alongside the fact that merely getting certain models to run is problematic. I am hoping that RNDA4 is paired with some tooling improvements. No more massive custom container builds, no more versioning nightmares. At my last startup we tried very hard to get AMD GPUs to work, but there were too many issues. permalink fedilink source parent
[–] vanderbilt@lemmy.world 1 point 1 year ago ROCm certainly has gotten better, but the weird edge cases remain; alongside the fact that merely getting certain models to run is problematic. I am hoping that RNDA4 is paired with some tooling improvements. No more massive custom container builds, no more versioning nightmares. At my last startup we tried very hard to get AMD GPUs to work, but there were too many issues. permalink fedilink source parent