The vRam for example limits you running local llms. Basically, the formula for t... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		exiguus 8 months ago \| parent \| context \| favorite \| on: NovaCustom – Framework Laptop alternative focusing... The vRam for example limits you running local llms. Basically, the formula for this is: 1GB of vRAM can handle 1B parameters. There are ways to overcome this, but the easiest way for the best performance, is just to have enough vRAM.

marssaxman 8 months ago [–]

If running local LLMs were a thing I ever wanted to do, I'm sure that would be something that mattered to me.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact