And even AMD is mightily struggling here, with hardware that is straight up bigger/faster than Nvidia.
The problem has always been software support. If Intel wants a piece of the AI pie, they need fantastic software support. AMD has always been a bit lackluster here, whereas Intel has done a pretty decent job in the past (esp. on Linux, their drivers rock), so they would need to double down if they truly want to get after it.
Intel is at their mercy
Then Intel should make their own laptops and prove the model.
it appears AMD is simply out designing them
I don’t think so, they’re just better at improving margins. Intel was able to keep up for a while despite not keeping up w/ the fabs, so I think their designs are absolutely fine. They’re not cheap to manufacture like AMD’s are, but they are really good.
Its all the delays! Its destroying them.
Exactly. They need to double down on something instead of faffing about with different ideas. Their money maker is server chips, so that should be top priority. Their next biggest is probably laptops, and AMD is getting massive inroads here due to Intel sucking on their fabs. Catching up on servers should be easier than catching up on laptops, because corps can be bought w/ value, whereas the CPU makes up a much smaller portion of overall laptop price, so they have less leeway here.
But yeah, they need to fix the delays. Get the fabs on track and get steady CPU production in their core markets. And do that without giving up on GPUs, because that needs to be in the future plans since people are generally moving away from CPUs to GPUs for compute.
Everything else Intel does can be scrapped for better software. Really good software can do a lot to make up for lagging hardware, so make sure that is top notch while you’re fixing the hardware delivery.
The problem has always been software support. If Intel wants a piece of the AI pie, they need fantastic software support. AMD has always been a bit lackluster here, whereas Intel has done a pretty decent job in the past (esp. on Linux, their drivers rock), so they would need to double down if they truly want to get after it.
Actually AMD is pretty okay for running LLMs and other ML workloads. Many libraries now explicitly target rocm, you can just plop down vllm or the llama.cpp server and have it work with big models out of the box. There are some major issues (like flash-attention), but its quite usable.
Intel though? Their software is a mess. You have to jump throigh all sorts of hoops, use ancient builds of pytorch, use their own quantizations and such to get anything working, fix Python errors, and forget about batched enterprise backends like vllm. And this is just their IGPs and Arc, forget trying to use the vaunted NPUs for anything.
This could change if they actually had a cheap 48GB GPU (or a big APU) for AI devs to target… But they don’t. And no one is renting Gaudi to build in support because its not even availible anywhere.
EDIT: oh, and one weird thing is the volume of Intel software support is high. Like they have all sorts of cool libraries, they make contributions to open projects… But its all disjointed and fragmented. Like theres no leadership or unified push, just random efforts flailing around.
Intel is shooting itself in the foot by going halfway. If they want to compete in the AI space, they need to go all-in w/ a solid software and hardware combo. But they don’t.
They have the capability, they’re just not focused. A good CEO should be able to provide that focus. Maybe they should hire Lisa Su. 😆
Speaking as an holder of AMD stock since ot was $8, and an all AMD CPU user, IMO Lisa Su is either an absolute idiot or colliding with her cousin, the CEO of Nvidia.
All they had to do was lift vram restrictions on consumer GPUs (so their OEMs could double the VRAM up) and sick like four engineers on bugs blocking the AI space, and they would be dominating the AI space and eating Nvidia’s pie…
And they didn’t. Like, its two phonecalls, thats it.
Intel had monumental problems it has to solve and struggles, but AMD has tiny ones they inexplicably ignore. Its mind boggling.
I work in CV and I have to agree that AMD is kind of OK-ish at best there. The core DL libraries like torch will play nice with ROCm, but you don’t have to look far to find third party libraries explicitly designed around CUDA or NVIDIA hardware in general. Some examples are the super popular OpenMMLab/mmcv framework, tiny-cuda-nn and nerfstudio for NeRFs, and Gaussian splatting. You could probably get these to work on ROCm with HIP but it’s a lot more of a hassle than configuring them on CUDA.
The problem has always been software support. If Intel wants a piece of the AI pie, they need fantastic software support. AMD has always been a bit lackluster here, whereas Intel has done a pretty decent job in the past (esp. on Linux, their drivers rock), so they would need to double down if they truly want to get after it.
Then Intel should make their own laptops and prove the model.
I don’t think so, they’re just better at improving margins. Intel was able to keep up for a while despite not keeping up w/ the fabs, so I think their designs are absolutely fine. They’re not cheap to manufacture like AMD’s are, but they are really good.
Exactly. They need to double down on something instead of faffing about with different ideas. Their money maker is server chips, so that should be top priority. Their next biggest is probably laptops, and AMD is getting massive inroads here due to Intel sucking on their fabs. Catching up on servers should be easier than catching up on laptops, because corps can be bought w/ value, whereas the CPU makes up a much smaller portion of overall laptop price, so they have less leeway here.
But yeah, they need to fix the delays. Get the fabs on track and get steady CPU production in their core markets. And do that without giving up on GPUs, because that needs to be in the future plans since people are generally moving away from CPUs to GPUs for compute.
Everything else Intel does can be scrapped for better software. Really good software can do a lot to make up for lagging hardware, so make sure that is top notch while you’re fixing the hardware delivery.
Actually AMD is pretty okay for running LLMs and other ML workloads. Many libraries now explicitly target rocm, you can just plop down vllm or the llama.cpp server and have it work with big models out of the box. There are some major issues (like flash-attention), but its quite usable.
Intel though? Their software is a mess. You have to jump throigh all sorts of hoops, use ancient builds of pytorch, use their own quantizations and such to get anything working, fix Python errors, and forget about batched enterprise backends like vllm. And this is just their IGPs and Arc, forget trying to use the vaunted NPUs for anything.
This could change if they actually had a cheap 48GB GPU (or a big APU) for AI devs to target… But they don’t. And no one is renting Gaudi to build in support because its not even availible anywhere.
EDIT: oh, and one weird thing is the volume of Intel software support is high. Like they have all sorts of cool libraries, they make contributions to open projects… But its all disjointed and fragmented. Like theres no leadership or unified push, just random efforts flailing around.
Exactly.
Intel is shooting itself in the foot by going halfway. If they want to compete in the AI space, they need to go all-in w/ a solid software and hardware combo. But they don’t.
They have the capability, they’re just not focused. A good CEO should be able to provide that focus. Maybe they should hire Lisa Su. 😆
Speaking as an holder of AMD stock since ot was $8, and an all AMD CPU user, IMO Lisa Su is either an absolute idiot or colliding with her cousin, the CEO of Nvidia.
All they had to do was lift vram restrictions on consumer GPUs (so their OEMs could double the VRAM up) and sick like four engineers on bugs blocking the AI space, and they would be dominating the AI space and eating Nvidia’s pie…
And they didn’t. Like, its two phonecalls, thats it.
Intel had monumental problems it has to solve and struggles, but AMD has tiny ones they inexplicably ignore. Its mind boggling.
I work in CV and I have to agree that AMD is kind of OK-ish at best there. The core DL libraries like torch will play nice with ROCm, but you don’t have to look far to find third party libraries explicitly designed around CUDA or NVIDIA hardware in general. Some examples are the super popular OpenMMLab/mmcv framework, tiny-cuda-nn and nerfstudio for NeRFs, and Gaussian splatting. You could probably get these to work on ROCm with HIP but it’s a lot more of a hassle than configuring them on CUDA.