But coreml utilizes ANE, right? Is there some bottleneck in coreml that requires... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		jvstokes 9 months ago \| parent \| context \| favorite \| on: Run LLMs on Apple Neural Engine (ANE) But coreml utilizes ANE, right? Is there some bottleneck in coreml that requires lower level access?

anemll 9 months ago [–]

Memory bandwidth is the main bottleneck. It got better with M3/M4. ANE is really fast in FLOPS but low in memory bandwidth.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact