LLMs are no different, and I can’t agree with you that open source models are not a threat to the big players.
This is just plain wrong. Again, there are zero open weight models that haven’t been developed by private companies. These companies, at the same time, offer superior closed source models because that’s their whole business model.
… we have an amazing amount of open source projects that people do simply because they want to. Those are the people we should support and the ones who freely train and fine tune open source models.
They are not.
There might be some people fine tuning models, but I can confidently assure you that there isn’t a single non profit entity out there that is spending tens of thousands of dollars in compute alone, just to give their model away for free. And that doesn’t even begin to account for data collection.
To your point about “software doesn’t become faster with time” mother fucker I remember windows 95, you’re delusional if you don’t think we’ve come an insane amount. I remember webpages taking minutes to load, interlacing vs non to help with image loading.
What are you even talking about. Websites in the 90s took longer to load because connections back then ran at 56Kbps tops, or ~5KBs, with latencies in the order of 500 to 1000 ms, when the average website would be like 10KB. Nowadays, an online newspaper weights 5 to 20 MB with average bandwidths of hundreds of megabits per second, with latencies of 50-100ms. Web development and its traversals are in such a particularly shitty state, browsing the modern Internet on less than 4GB of RAM is borderline impossible. In other words, software has become slower, and hardware is doing the heavy lifting now. And I can say this because I work in the field.
Phones require specialized hardware and designing, to run and produce, LLMs only require normal consumer grade hardware and the desire to learn how to make it work.
This is such a massive mischaracterization.
First of all, it’s easier to put a phone together with off the self parts, than it is to build a meaningfully useful LLM even with $50,000 worth of hardware at one’s disposal. Second, running a LLM was never the issue. Being able to produce and run a meaningfully useful LLM that has no strings attached to private interests is.
Honestly, I think you are out of your depth. Being a hobbyist is fine, but holy crap please inform yourself. None of this shit is easy or free or even cheap to build and run, and every foundation model is controlled by private interests.
This is just plain wrong. Again, there are zero open weight models that haven’t been developed by private companies. These companies, at the same time, offer superior closed source models because that’s their whole business model.
They are not.
There might be some people fine tuning models, but I can confidently assure you that there isn’t a single non profit entity out there that is spending tens of thousands of dollars in compute alone, just to give their model away for free. And that doesn’t even begin to account for data collection.
What are you even talking about. Websites in the 90s took longer to load because connections back then ran at 56Kbps tops, or ~5KBs, with latencies in the order of 500 to 1000 ms, when the average website would be like 10KB. Nowadays, an online newspaper weights 5 to 20 MB with average bandwidths of hundreds of megabits per second, with latencies of 50-100ms. Web development and its traversals are in such a particularly shitty state, browsing the modern Internet on less than 4GB of RAM is borderline impossible. In other words, software has become slower, and hardware is doing the heavy lifting now. And I can say this because I work in the field.
This is such a massive mischaracterization.
First of all, it’s easier to put a phone together with off the self parts, than it is to build a meaningfully useful LLM even with $50,000 worth of hardware at one’s disposal. Second, running a LLM was never the issue. Being able to produce and run a meaningfully useful LLM that has no strings attached to private interests is.
Honestly, I think you are out of your depth. Being a hobbyist is fine, but holy crap please inform yourself. None of this shit is easy or free or even cheap to build and run, and every foundation model is controlled by private interests.