I mean, it's not completely fatal, but it means an approximately 16x increase in... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		lechatonnoir on Oct 24, 2024 \| parent \| context \| favorite \| on: Probably pay attention to tokenizers I mean, it's not completely fatal, but it means an approximately 16x increase in runtime cost, if I'm not mistaken. That's probably not worth trying to solve letter counting in most applications.

SEGyges on Oct 25, 2024 [–]

it is not necessarily 16x if you, e.g., decrease model width by a factor of 4 or so also, but yeah naively the RAM and FLOPs scale up by n^2

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact