I didn’t say you can’t have a GPU, but to me, its wasteful. I keep my jellyfin server off when not in use, and use WoL to start it when its needed.

I have played with local LLMs, and the models I used were unimpressive, but without knowing what the OP has in mind, we cant know how much power it will use. If it just spins up the GPU once a day for 20 minutes, probably okay, you won’t even notice it. But anyone like me who doesn’t already have a GPU in their lab will probably notice it quite clearly on their power bill.

A megacorps server farm is huge, but its also amortised over millions of users, they probably don’t need 1-1 GPU to customers, so the efficiency isnt necessarily bad. (Although at the moment, given megacorps are tripping over themselves to throw compute at LLM training, this may not be true)

Idle is low power, not zero power. And it won’t be idle when its scraping and parsing the sites, so depending on how much scraping its doing, it could be significant non-idle energy usage.

Yeah, absolutely. And running a GPU 24/7 to occasionally search is just a waste of power. I’m not convinced that google and bings AI search makes financial sense either, Google dropped live search (where the results updated as you typed realtime) because it was too expensive, how does LLM search end up cheaper than live search?!

Edit: This is the live search thing: https://searchengineland.com/test-google-updating-search-results-as-you-type-49116 ~~Annoyingly hard to find, and I can’t find the articles on its cancellation, but from memory it was related to expense. ~~

Edit2: Google Instant Search, and the death was blamed on mobile, and wanting to unify the mobile/desktop experience. I do vaguely remember expense being an unofficial/rumored reason, but I can’t back that up.

I personally have zero interest in AI search, if you mean LLM. The fact that it can make stuff up, also means it can miss stuff as well. Neither are acceptable for a search engine.

If you mean some kind of deterministic algorithm for indexing and searching, then maybe.

Also, attempting to crawl sites locally sounds like a great way to get banned from those sites for looking like a bot.

My repo is fine, because its not online. Keeping them online is just silly.