Yeah I did forget to consider that a lot of the web hosted models have a whole array of “experts” - Sub-LLMs that help fill in specialized information that a more generalized LLM wouldn’t have. Not a problem for someone running an AI model on their home computer but something that likely happens most times that you’re querying an AI online.
That’s also true, though it’s important to remember that the “experts” aren’t experts in the classical sense. Say you have a word made up of 3 tokens, it’s possible that each token is routed to a different expert. It’s just a model architecture.
While the order of magnitude is correct, running the bigger models is closer to playing a AAA game on 8 computers at the same time.
Yeah I did forget to consider that a lot of the web hosted models have a whole array of “experts” - Sub-LLMs that help fill in specialized information that a more generalized LLM wouldn’t have. Not a problem for someone running an AI model on their home computer but something that likely happens most times that you’re querying an AI online.
That’s also true, though it’s important to remember that the “experts” aren’t experts in the classical sense. Say you have a word made up of 3 tokens, it’s possible that each token is routed to a different expert. It’s just a model architecture.