Good tips - thanks!
PS: sad to report the 24GB Tesla p40s are now around $250 USD on eBay, so not quite as cheap as I remembered. P4s are still cheap tho, though frankly if you’re going that end of town, a 1080 is about on par, less fussy and probably cheaper - it just won’t fit in a uSFF.
You probably could. A Tesla P4 or P40 (old data centre cards) are more than up to the job. My Lenovo tiny hosts a P4 (card cost $100 on eBay; the lenovo itself was $200ish) and runs Qwen3.5-35B-A3B at about 20 tok/s. Smaller models are even faster.
https://www.youtube.com/watch?v=8F_5pdcD3HY
If you’re not bound by the one liter shoebox design, then the P40 is still a great and inexpensive card.
I think I mentioned elsewhere but right now I’m trying to figure out if I can use a magic packet from the Raspberry Pi to wake up the Lenovo as needed rather than leaving it on all the time.
Agree. I know the Pi’s are out of favour these days…but they are a cool little machine. I got mine running DietPi and a bunch o crap (the usuals - JF, arr stack, pi hole, syncthing, yadda yadda) and running headless the footprint (power and memory wise) is tiny.
I joked about the 4xAA batteries thing but iirc, there is actually a Pi-HAT that creates a micro UPS that’ll run the pi for maybe three to five hours just on double A batteries.
Edit: yep
https://pimodules.com/product/ups-pico-hv4-0-advanced
or more sensibly
Agree. And re small models - very agree. In fact I made a ablated version of Qwen 3.5-2B for use with my pi, before thinking a bit harder and realising I can probably code something bespoke that doesn’t need a stochastic parrot as a squwake box at all.
https://huggingface.co/BobbyLLM/polaris-heretic-Q4_K_M-GGUF
Still, as a SLM, it’s perfectly cromulent and does well with tool calling etc which is what I wanted it for.
There’s an argument to be had regarding a MoE versus a small dense model. I guess it depends on what exactly you need doing with it. I would be tempted to run a smaller dense model (like a Qwen 3-14B or a Qwen 3.5 9B) as at a reasonable quant, it might fit mostly or entirely on the GPU, thereby giving you excellent speeds.
PS: I’m actually in the process of designing an expert system (not a LLM) for pretty much the task you described. The intention is that you would still interact with it like a large language model, but the actual brains underneath it would be something more traditional.
Which trackers if you don’t mind saying? DM me if easier.
Yep. But that would be 100% CPU, 100% of the time? Real life, it’s probably closer to 2w idle and maybe 5-7W under typical load.
More interesting…I think that technically means you could make a “UPS” for it using what…4xAA batteries?
Oh man…that would be cool. Stupid but cool.
They were, I think. Or we were just younger.
Yeah, same. Though at 3-5W … it really is just a very rough guess. Lemme ShitGPT it. Oh, I was way off
A realistic Pi 4B-only estimate is about A$8–A$12 per year in electricity, assuming it is on 24/7 and used for Jellyfin streaming around 10–12 hours per week.
Pi 4B measurements are typically around 2.7–2.85 W at idle, about 5.1 W under moderate server load, and around 6.4 W under full CPU stress. Using Perth/WA’s Synergy Home Plan A1 energy charge of 32.3719 c/kWh, excluding the daily supply charge, that works out very cheaply because the device uses only about 25–36 kWh/year.
Scenario Assumed usage Annual energy Approx. annual cost
Mostly idle 3 W 24/7 26.3 kWh A$8.51/year Idle + 12h/wk Jellyfin 2.7 W idle, 5.1 W streaming 25.1 kWh A$8.14/year Heavier Jellyfin/server use 2.7 W idle, 6.4 W streaming 26.0 kWh A$8.40/year Conservative wall-power estimate 4 W idle, 6.4 W streaming 36.5 kWh A$11.83/year
The bigger swing factor is storage, not the Pi. A USB SSD adds very little; a USB-powered 2.5" hard drive might add a few dollars per year; a powered 3.5" external drive left spinning 24/7 could push the total more into the A$15–A$30/year range.
So, for the Raspberry Pi 4B itself as a Jellyfin box: roughly A$10/year is a good mental estimate.
I remember it being a touch more …analog…back in the day. ATDT commands and all.
But yeah, Win 3.11+ trumpet winsock and Free Agent were the shit. Rec.martial.arts was home back then (along with mIRC).
Lemmy reminds me a bit of the old Usenet fora.
Torrent cache? As in seedbox?
Use to last me 2-3 months… but my media library is more or less complete now, with little churn. Also, I don’t ever go above 1080p.
I need to check if Radarr / Sonarr works with straight torrents (it must do; I haven’t used them for ages / have been using 1337 manually, but I seem to recall torrents being a source).
Debatable :) Torrents rely on seeders. I’ve downloaded movies and TV shows >5 yrs since initial upload via Usenet. Yes, things expire there too (eventually), but when the getting is good, it’s uniformly good / fast.
OTOH, 1337 has been pretty decent to me of late.
It’s tricky. On one hand, Jellyfin and the arr stack are what got me into self hosting. OTOH…torrents are simpler - I can plug my external SSD directly into my router, which streams to NovaPlayer on any android device - nothing else needed. Want a new show / movie? Grab the torrent, punt it across to ssd via samba share. It auto populates.
https://github.com/nova-video-player/aos-AVP
It’s…simpler. Arguably more elegant / less moving parts.
Dunno.
Yarrr! But it really mostly is Yarr these days. So don’t go firing up Trumpet winsock to check Forte Agent :)
I was tempted to say $0, but then I thought harder about the problem.
Technically I do have ongoing costs
https://usenet-news.net/index1.php?url=home
Electricity (whatever tiny amount raspberry pi sips). At a guess, maybe $50/yr.
So, amortised over time - very low but not zero. In theory, if I dropped Usenet, it would even lower. And theoretically, I could run the pi off a single solar panel and a diy solar kit but I’m not busy pretending to be Robinson Crusoe just yet. Though… It might be a cool project.
No good deed goes unpunished. The sense of self entitlement some people display is staggering. FOSS project? Well, you should have done x y or z.
Also, I gave you $3 via Ko-fi, so you need to provide customer support in perpetuity and come to my house and install it. And heaven forbid you try to recoup costs!
Projects don’t just die out - a lot of them are killed (one way or another). For example, I had a fully specced out FPGA design that would capture the signal from Wii GPU and do internal upscaled resolution (think: like what dolphin emulator does but with actual hardware) not just post process sharpening. Total cost under $100 and some know how.
The amount of flack I copped for it made me shut down the github and work on it for myself. Once it’s perfected, I may post about it again but I sure as shit am not compelled to deal with the fucking peanut gallery anymore.
The Jellyfin vs Plex thing always struck me as odd. As in - why are we holding JF to a different standard to (say) Immich, Syncthing, Pi-hole or any one of a thousand different programs people self host?
Yes, JF ships multi-user accounts and client apps etc. I get it, “multi-use” is implied, so the comparison isn’t totally unfair. But there’s a difference between ‘this feature exists’ and ‘this is the primary purpose of the tool’.
The fact that you CAN share it externally doesn’t mean everyone running JF is doing that, or that it should be the benchmark the whole project is judged by.
To me, self host means “I host it, myself” not “I host it and then pretend to be Netflix for family and friends”. If that’s the use case, then of course, Plex away.
It’s cool that you CAN share JF externally, and it’s cool that Plex does that differently / better. We shouldn’t hold one to the standards of the other.
There use to be one called PDFDRIVE. I mean, there still is, but there use to be too :)
Huh - cheaper than the P40s (though less VRAM) but larger bandwidth due to HBM2. Good looking out