I personally have 4 boxes stacked in a corner that are connected to a "rack", which is just two rack side panels bolted to the back of a workbench that I screw components into, connected to NAS and multiple Raspberry Pi's on a shelf and I really like the mess and have learned a lot.
Just use what you have and expand as you need. Racks are cool once you're into the hobby enough to care about style points.
I got a few recently, and they're just great, particularly in terms of power usage. I hooked mine to tasmota-powered smart plugs, and they idle at something like 6W ... Granted, I always tune hardware (from bios config) and software (tuned profile) for low power... But long story short, my nucs rarely spike over 30W.
Literally half of one of those old tungsten-based lightbulbs.
You can buy old used laptops (especially with broken monitors) for basically nothing (or maybe literally nothing if you already have one, or you have a friend of family member who has one they're willing to donate). If your goal is just to use it for Plex, it will work fine, and the power draw on laptops (particularly when idle) isn't that much more than a NUC.
I use a proper rack mount server now largely because I thought they were neat, but I think for most people an old laptop is fine.
That being said, when I finally got around to rackmounting and upgrading some of the other hardware in my lab, this "beginner"'s guide was really helpful.
It's not the most sophisticated setup on earth, but it works great, it's dirt cheap, and it's apartment-friendly.
I used to use a half rack full of Xeon based Dell and HP blades - but the blade chassis are huge/heavy and the power usage was HUGE!
I don't need Proxmox, tons of containers or anything. If I do, I can always throw a N95 into the mix with ample RAM and storage.
Additionally, one challenge I have with all of these is that none of them have anything like a BMC. If you have more than one system, moving peripherals around sucks, and KVM technology is kind of iffy these days. It seemed easier to just jam an old Supermicro motherboard that had IPMI enabled in a reasonable ATX case with a lower power Xeon variant, and call it a day.
[1] https://thin.computer/index.php/2024/03/08/self-hosting-my-s...
Proxmox makes it very easy to have multiple containers and VMs on a single hardware device. I started by just virtualizing the one big Ubuntu system. Even that has advantages: backups, high availability, etc. But now I'm starting to split off services into their own containers and it's just so tidy!
I find myself referring to my git repo frequently as a source of documentation on what's installed and how the systems are set up.
For a few years I put mine inside an IKEA FRIHETEN sofa (the kind that just pulls†).
Pros:
- easy access
- completely invisible save for 1 power wire + 1 fiber (WAN) + 1 eth (LAN)††
- structure allows easy cable routing out (worst case you punch a hole on the thin low-end)
- easy layout/cable routing inside
- noise reduction for free
- doubles as warming in winter a seated butt is never cold
- spouse enjoys zero blinkenlights
- spouse didn't even notice I bought a UPS + a disk bay
Cons:
- a bit uncomfortable to manipulate things inside
- vibrations (as in, spinning rust may not like it) when sitting/seated/opening/closing
- heat (was surprisingly OK, not worse than a closet)
- accidental spillage risks (but mostly OK as design makes it flow around, not inside the container, worst case: put some raisers under hardware)
- accidental wire pull e.g when unsuspecting spouse is moving furniture around for housecleaning (give'em some slack)
† https://www.ikea.com/us/en/images/products/friheten-sleeper-...†† that I conveniently routed to a corner in the back then hidden in the wall to the nearest plugs, so really invisible in practice.
Everyone has some set of goals... low power, interesting processors, data ownership, HA, UPS/whole home UPS... and the home is the only common intersection of all these overlapping interests, and bits of software. Even more fascinating is the types of people it attracts, from professionals playing to people far outside the industry.
I have really taken the plunge and it recaptures some of the magic of the early internet (at least for me).
* Location: Sitting on a shelf in my basement office. Ventilation is okay, the WiFi is fine but not great.
* Hardware: An old PC I picked up at a neighborhood swap meet. I added some RAM taken from another old PC and bought a hard drive and WiFi card.
* Software: Debian stable and podman/podman-compose. All my useful services are just folders with compose files. I use podman-compose to turn them into systemd units.
If the stuff in the article is the kind of thing you're into, that's awesome, go ham! But you absolutely do not need to ever, and you certainly do not need to do it right away. I run a bunch of services that my family uses daily on this thing, and we use less than half the 16GB RAM and never get over 5% CPU usage on this old, ~free PC.
- My real problem is disaster recovery. It would take me forever to replicate everything, if I could even remember it all. Router configurations, switch configurations, NAS, all the various docker containers scattered across different vlans, etc. I mapped out my network early on but failed to keep it up to date over time. Is there a good tool to draw, document, and keep up-to-date diagrams of my infra?
- Backup and upgrading is also a persistent problem for me. I will often set up a container, come back to it 6 months later, and have no idea what I did. I have dozens of containers scattered across different machines (NUCs, NAS, desktops, servers, etc). Every container service feels like it has its own convention for where the bind mounts need to go, what user it should be run as, what permissions etc it needs. I can't keep track of it all in my head, especially after the fact. I just want to be able to hit backup, restore, and upgrade on some centralized interface. It makes me miss the old cattle days with VM clone/snapshot. I still have a few VMs running on a proxmox machine that is sort of close, but nothing like that for the entire home lab.
I really want to get to a point, or at least move towards a solution where I could in theory, torch my house and do a full disaster recovery restore of my entire setup.
There has to be something simpler than going full kubernetes to manage a home setup. What do you guys use?
I’ve learned so much about containers, virtual machines and networking. Some of the self hosted applications like paperless-ngx [1] and immich [2] are much superior in terms of features than the proprietary cloud solutions.
With the addition of VPN services like tailscale [3] now I can access my homelab from anywhere in the world.
The only thing missing is to setup a low powered machine like NUC or any mini PC so I can offload the services I need 24/7 and save electricity costs.
If you can maintain it and have enough energy on weekends to perform routine maintenance and upgrades. I would 100% recommend setting up your own homelab.
[1] https://docs.paperless-ngx.com/ [2] https://immich.app/ [3] https://tailscale.com/
As an apartment-dweller, for a long time I had it in a closet. Once I moved it to living room, my solutions included:
* For discreet, IKEA CORRAS cabinet that matched my other furniture. I had rackmount posts in it before, but got rid of them because they protruded.
* For noise, going with gear that's either fanless, or can be cooled with a small number of Noctua fans. (I also replace the fans in 1U PSUs with Noctuas, which requires a little soldering and cursing.) I tend to end up with Atom servers that can run fanless in a non-datacenter except for the PSU.
* Also for noise, since my only non-silent server right now is the 3090 GPU server, I make that one spin up on demand. In this case, I have a command I can run from my laptop to Wake-on-LAN. But you could also use IPMI, kludge something with a PDU or IoT power outlet, find a way to spin down the 3090 and fans in software, make Kubernetes automate it, etc.
* For lights, covering too-bright indicator LEDs with white labelmaker tape works well, and looks better than you'd think. Black labelmaker tape for lights you don't need to see.
* For console, I like the discreet slide-out rack consoles, especially the vintage IBM ones with the TrackPoint keyboards. If I was going to have a monitoring display in my living room, I'd at least put the keyboard in a slide-out drawer.
* I also get rid of gear I don't need, or I'd need more than twice the rack space I currently have, and it would be harder to pass as possibly audiophile gear in the living room.
* For an apartment, if you don't want to play with the router right now (just servers), consider a plastic OpenWRT router. It can replace a few rack units of gear (router, switch, patch panel), and maybe you don't even need external WiFi APs and cabling to them.
Why do I have a home network? Like many here, I develop network applications. I'm not a network technician, but being able to truly understand how the various components of the internet work, like TCP/IP, DNS etc., is really useful and sets me apart from many developers. I also just like being in control and having the flexibility to do what I want with my own network.
Why do I have multiple computers? Playing with different operating systems etc. isn't really the reason as virtualisation is pretty good these days. It actually comes down to locality of the machines. I want my hard disks to be in a cupboard as they are quite noisy, but I want my screens and keyboard to be on a desk. So I've got a NAS in a cupboard and a (quiet) PC at my desk, plus a (silent) media centre in my living room and other stuff.
One thing I would say is don't be tempted by rack mounted server gear. Yeah, it looks cool and you can get it second hand for not much, but this isn't suitable for your home. Use desktop PC cases with big fans instead. Rack mounted network gear is pretty cool, though.
Try to find one of these hundreds of small form factor products that has anything faster than a 2.5Gbit nic and despair! What good are these machines if you cant even get data in and out of them?!
The list of hardware with 2x10GBe or better connectivity is amazingly short:
- Minisforum MS-01
- GoWin RS86S*, GW-BS-1U-*
- Quotom Q20331G\*
From major manufacturers, only the following: - Mac Mini (1x10gbaseT option, ARM, )
- HPE Z2 Mini\* (1x10GbaseT option)
Please someone, anyone, build a SFF box with the following specs: Minimum of 2 NVMe
Minimum of 2 additional storage devices (SATA or more NVMe)
Minimum of 2 10GbE
Dedicated or shared IMPI/OOBM
Minimum of 8 Core x86 CPU
Minimum of 64GB RAM
Make sure the nics have enough PCIe lanes to hit their throughput limits, give the rest to storage. Stop with the proliferation of USB and thunderbolt and 5 display outputs on these things, please already!This is the way. HP, Lenovo, and Dell are all making identical small form factor PCs that are more than enough for most people at home. They are also very silent and power efficient.
I have a Lenovo M720 Tiny and a Dell Optiplex 3080 Micro (they are virtually almost the same). You can change parts, there are ample ports available and you can pretty much run any OS you want.
Was hoping this article would delve into hosting your own email, websites and services and the steps to expose them to the public internet (as well as all the requisite security considerations needed for such things).
A homelab is just a normal PC unless you're doing homelab stuff.
"Be aware, the video server and sync server will be offline between the hours of 8PM and 10PM on Friday, March 8th for hardware upgrades. Expected downtime should not be more than 20 minutes but please plan around this disruption to essential services. Other services should not be affected during this work."
Like someone else mentioned in this thread: It was really best described as a "home network with more than one computer." But you can't really put that on your C.V. under "experience."
"Improved SLA uptime on client's network to five nines. Upgraded storage controllers in a live environment without disruption to established services. Deployed k8s cluster into a production environment serving thousands of hours of compressed video files with real-time format transcoding."
I will freely admit, my home network probably needs to be pared down a bit. The problem is the salvaged hardware and good deals at the liquidation auctions accumulates faster than I can cart it off to the street corner. There's also far too many laptops floating around the house doing nothing useful that could be donated away. A lot of the homelab is salvage or bought at (in-person) liquidation auctions. There's some new stuff, but I rarely need the latest and greatest for my setup. In my home network diagram I try to document everything that touches the network so I can stay on top of the security patches, and what has access to the outside world and in what way.
https://justinlloyd.li/wp-content/uploads/2023/11/home-netwo...
> Pros: Less noise, easier cable runs.
> Cons: can get hot depending on where you live, roof leaks, humidity/condensation, and creepy at night.
Where do attics not get hot? I live in northern climate, and ours regularly gets hotter than 140F during sunny summer days.
And I say hotter, because the remote temp sensor I have up there maxes out at 140F. I wouldn't be surprised if it actually gets up to 150F or more.
No reason to go crazy with your home server if it is only you using it. You can get away with a crappy laptop that your roommate is done with.
It allows you to go deep into settings and configure anything to your heart's content. But, if you want to ignore it for a year and not mess with it, it will run as smoothly as an Eero or a similar consumer product like that.
I have a Purple model[3]. I initially balked at the price, but it performs well, and there hasn't been a single day in the past couple years where I've regretted it or looked at something else.
It supports just about any router feature. Has robust and easy to configure device/family profiles. The quality of the Firewalla software and the mobile app is excellent.
---
1. I manage a quite large WAN during the day. I don't want to do it at night, too.
Nearly everyone has a spare PC lying around collecting dust, and within an hour it can be running a dozen VMs . Reclaimed HP & Dells are cheap and include a Windows Pro license. otherwise they can be had for $20
I recommend Alpine for the guest OS. It's fast to set up, natively supports Hyper-V hosts and compact (about 1/10th the size of a debian guest).
https://learn.microsoft.com/en-us/windows-server/virtualizat...
For networking I can recommend Fujitsu Futro S920 thin clients, especially for EU users. They are dirt cheap here, and with aftermarket riser & intel NIC they are great routers with opnsense/pfsense. Apart from that, I have a couple of raspberry pi's running dedicated tasks, and space heaters (e-waste tier enterprise desktops) running BOINC during coldest months.
There are just SO MANY perfectly good computers that are bound to be e-waste that would be good for this.
* Hardware: A regular but small (SFF) computer + Synology NAS
* Software: Ubuntu distro + Docker containers bare
* Configuration: Git-versioned Dockerfiles and docker-compose.ymls
* Storage: The NAS with Synology Hybrid RAID
* Backup: TBD... haven't figured out a cheap and automatic way to backup 20 TB of data off-site
I run cameras, a media server shared with friends, a shared place to dump project files, a DNS server (just in case the Internet goes down so that everything local still works) and supporting services.
The best I've come up with so far is segmentation via VLANs, then adding many rules to deal with inter-VLAN reachability (e.g. should it be forced to transit to the firewall and back? Can my laptop reach out to the IoT devices, even if I block connections in the reverse direction?).
Beyond this, I'm not sure how to best handle similar segmentation on my few servers (all running proxmox), e.g. whether to expose VLANs to the VMs (which I kinda need to do for the pfsense firewall, maybe?).
The biggest thing is how to keep these network configs in sync, e.g. when adding a new VLAN, etc. I basically have an Org file that I write all this down, but I'm guessing there's a better way.
Also, if anyone wants a cheaper alternative to Ubiquiti, Mikrotik has served me well for years. I'm considering switching to Ubiquiti stuff though, as they seem to have a solution to synchronize state between multiple switches that is much better (vs. manually using Winbox on each individual router).
The quality of software out there these days is insane -- there are projects that cover so many crucial functions (tracking expenses, playing music, managing data, etc), that it's becoming really hard to track them all.
I've been working on converting the site to a directory and it's not quite there yet -- but some people do like the massive wall of projects.
I have a beginner setup: a single NUC and a cheap NAS with NFS.
I have yet to come across something that this setup cannot handle.
When I saw the picture of the server mounted in a cardboard box, my first thought was about fire hazards. It turns out that might not be as big an issue as I initially thought, since paper won't spontaneously catch fire until reaching ~200C [1], and the operating temperatures of computers generally shouldn't exceed 100C. However, I'd still be a little wary about a cardboard setup, considering the potential for electrical faults, the heat generated by power supplies and the risk of inadequate ventilation leading to hotspots. These factors could present a significant fire risk. My initial judgment of the setup may have been too quick but not entirely unfounded I guess!
I have mostly consumer (non-rack) equipment in a small Ikea closet. Looks less nerdy, which also improves the WAF.
These are mostly 1st-gen i7-4600U Haswell's, upgraded to 16GB RAM and 128G+ SSDs. I ripped them out of their cases, added heatsinks to the new SSDs, and then "racked" them by wedging the front-bottom mainboard corner (if plugs are oriented to the rear and the heaviest plug, AC, at the rear-bottom for balance) into a slotted piece of super-strong and right-angled cardboard that came from inside some re-inforced shipping box (I think it was an X-mas tree).
I assembled all of this juuuuuuust before COVID while learning K8s and craving the physical plan aspect, as I had just transitioned to a remote SaaS role from one that involved multiple physical DCs and travel.
My "go small lab" initiative turned into some sort of Tribble fable.
Why are so many different components needed?
This is the biggest thing for me, I would love to have a big fancy lab setup at home, but when you start adding up the power consumption and put a monthly dollar amount to it, then it becomes less exciting. Especially if you live somewhere like California where home electricity bills are incredibly high nowadays, it is extra painful.
There are some low-power setups that are great, but it seems like once you start using rack-mounted equipment, the machines get more power-hungry.
TF to bootstrap vms + rke cluster. TF to deploy helm charts. Still some per app config/backups to mange but I just copy container volumes to all the machines in the cluster with the csi plugin and then back everything up to zfs nas via truenas on NFS.
Runs primarily on 3 dell rx30 series, and then my desktop and htpc are also in vsphere running gpu centric things when I'm not gaming with them.
Vmug is pretty perfect for this -> $180/yr = 16 or 36 physcial sockets.
[1] https://us.ecoflow.com/products/river-2-pro-portable-power-s...
I considered moving them to the garage and then running USB-over-fiber to access them but have a ton of flight sim gear I need powered as well.