Big and not so big technologies in a small house

In this article I will try to summarize my personal experience that has accumulated in the process of creating infrastructure 'for the home' with a description of the technologies used and proven to be effective.

Iron

Routers – Mikrotik. With a fairly adequate price tag – a bunch of features and a wide variety of hardware, which is updated quite quickly both in terms of line and software. And in terms of features. And if the CRS125-24G-1S-2HnD switch has been around for a long time and suits everyone, then the routers are updated periodically. In this case, the new router is installed on the main provider, now it is C53UiG + 5HPaxD2HPaxD with WiFi-6, and the old router moves to the upstream to the backup provider. Yes, Mikrotik is not ideal, yes, there are some difficulties with transferring the config from the old hardware to the new one, but the advantages outweigh. And the advantages for me are as follows

Scripts. Email notifications about new versions, automatic backups – all this is done through them.
Supports a bunch of VPNs in both directions
Remote logging
Choice of CLI/API/Web interfaces
Flexible routing of individual networks (if you know what I mean)
Constantly increasing functionality and performance characteristics, for example, the C53UiG+5HPaxD2HPaxD already has containers and a gig of memory

Server – Mini-ITX with 2 SATA and M.2. Pros

Silence. For example, I have a J5040-ITX, it has no active cooling on the processor, it is in an ITX case Thermaltake Suppressor F1 with a large fan (200 mm) across the entire case, which, due to its size, is not particularly noisy but blows everything inside.
Flexibility in storage. Two SATA – for RAID 1, M.2 for the operating system and data that you don't mind losing.
Two Dual Channel DDR4 slots. I currently have 8G there, more than enough, 2 are in use and 5 are for caches.

Single-board computers – a universal thing. I have one (Raspberry 4) looking out from Nexcloud, git in the form gogsand reverse proxy in the form caddy before all this. The other one (Raspberry 3) is working under octoprinter to control the 3D printer, the third (Raspberry 4) works as a server home assistant. For all sorts of fun like kubernetes, there is a certain number of machines on orange (zero 2 w, pi 5). Thanks to Linux, Ubuntu and Docker, everything is installed and configured on them exactly like on servers with x86_64. TTX is also growing. Orange PI 5 is 8 cores and 4/8/16G of memory, by the way.

Technologies and software

IaC (Infrastructure as Code)

The basis of the basics. Well, after Linux. The beginning of the implementation of this technology at home was laid by the fight of RKN with Telegram, when entire subnets were banned, and the old technology – 'I'll go via ssh, configure everything and set up automatic updates' did not work very well, firstly, the servers had to be changed, and secondly, the amount of software on them grew. Then, two years ago, trust in clouds fell sharply, a server on an ITX board appeared at home and now (since May 2022) the number of roles in ansible reached 27. There is no point in editing the config on the server when you can store it in git and deliver it to the server via ansible. In general, this is implemented on the following crap:

Ansible – in 99% of cases, basic modules that are quite easy to understand will be enough for home use. Also, large language models will help you, they can do ansible, some worse, some better, but for starters, they are a great help. Flexible, fashionable, youthful. A new server is filled with the necessary software in the necessary configuration easily and naturally after adding it to the inventory. For servers, I use Ubuntu, it works great on both arm and x86_64, there is no difference for ansible roles. Flexibility – through roles and tags. All sensitive data (keys, passwords, addresses) can and should be encrypted and then all the code can be stored even in github. Many examples are easy to find.

Git – Store ansible code and mirror your projects on github. I use gogs on Raspberry 4, which looks out. UI on https, ssh, all the necessary minimum. Deployed as a docker container with 31 lines of code in ansible. Half of which is pushing the config caddy which works as a reverse proxy with https termination. One of the downsides for me is that gogs can't pin closed repos from github. But this is easily solved by adding `git remote` and the habit of doing `git push && git push mygit`

Docker – works great on arm and x86_64. Greatly simplifies software installation. With docker-compose, you can create and manage entire groups of services. Easily managed via ansible. The only problem for me is that containers do not fall into the scope of unattended-upgrades from ubuntu, and containers sometimes look outside. I solved this problem for myself using watchtower.

Monitoring

I started quite a long time ago with light monitoring like cacti And munin-monitoring. When I started the server, I built monitoring on my habit zabbixthen I tried prometheus and switched to it. Just yesterday I removed the zabbix roles from ansible because I definitely don't plan to go back to it.

Prometheus, Grafana, Alertmanager, Node-exporter – are installed perfectly via docker-compose and ansible. Everything is beautiful, easy to configure, lots of examples. Node-exporter is installed on arm and x86_64, easy to configure for tls and password. Eats up fewer resources than zabbix + mysql. Writes alerts to Telegram. Collects metrics from homeassistant.

Blackbox_exporter – easy to install via docker-compose and ansible. Easy to configure to work with Prometheus. I monitor the main http servers in RuNet and beyond for a general understanding of their availability and access speed.

mktxp – a ready-made solution for monitoring Mikrotiks via Prometheus and receiving logs from them in Loki with display in Grafana. The examples include a setup and a docker-compose file. It works great via Ansible.

cadvisor – since there are already 13 containers on the main server, I use it to monitor them. It also works via Prometheus. Also via docker-compose and ansible.

netdata – beautiful, easy to install on everything, but I think I'll give it up because Prometheus covers all my monitoring needs.

Anything else

Web / Reverse Proxy – caddy. One binary, available for all used architectures, out of the box letsencrypt with minimal config (I don't like nginx for a bunch of configs), works great as a reverse proxy for virtual hosts. For home with IaC – it's the best.

Network storage – nextcloud. Not an ideal and heavy solution, but there are clients for all platforms, it can handle standard protocols like WebDav, there is a UI. In general, it is quite ok to save or share something. It works quite well on Raspberry 4 with 4G RAM.

Distributed storage – syncthing. Easily controlled via ansible. Can encrypt where necessary (for example, on a remote virtual machine) and not encrypt where not necessary. Now it works quite successfully synchronizing one folder on two Androids, two Macs, 5x Linux (Ubuntu and Gentoo) and one IPad. Generally uses publicly available ones for synchronization behind NAT relaysbut if you are paranoid, you can build the infrastructure exclusively on your relays. It does not eat up the battery on your phone significantly. One minus – on IOS/IPadOS the client is third-party and paid (but not very expensive).

Notes – Obsidian. Lots of plugins, clients on all used platforms, lots of information, one minus – not opensource. Despite my recent article, after some experience of operation, I do not recommend using git for synchronization, this plugin chokes with a large number of files. But in conjunction with syncthing works great. Thanks to the commentators of the article for getting me interested in tinkering with syncthing.

Smart House – home assistant on zigbee. Now it works on Raspberry 4. Unfortunately, it has no IaC, but there are backups. In fact, the installation has already survived 3 servers, Raspberry 2->3->4. Lots of integrations. With Alice, with Mi Home, with Prometheus, with a toothbrush. The toothbrush is no joke, there is quite an integration with OralB.

Passwords – keepass. Clients for everything, only for IPad it is crooked (or I just found such), it can't normally connect with nextcloud. On Android/MacOS/Linux/Win everything is great, in conjunction with nexcloud it works for a long time and great. I think that with syncthing there won't be any problems either. And perhaps I'll switch to it soon myself.

Photo/video storage – photoprism. Actually, almost nothing has changed since the article on this topic was written almost two years ago. It works great via docker-compose and ansible. The number of photos and videos has grown to 296G. The free version finally has maps by geotags. One thing is annoying – the strange behavior of the UI when used from a phone. Sometimes scrolling is reset, sometimes instead of viewing – selection works. But so far I don't see any alternatives.

Backup – restic. Great tool, encryption, incremental backups, lots of places where it can be stored, easy recovery, easy to use. I recommend it. Easily managed via ansible.

UPS – nut. A bunch of supported UPS, there are remote clients, easy to manage via ansible.

Blog – hugo. Easy to store in git, built-in github.io integration, very easy to move to any place where there is a web server. The source format is quite convenient MD.

Some pictures

Loading the server Intel(R) Pentium(R) Silver J5040 CPU @ 2.00GH with 8G RAM on which monitoring with a logon acceptor, photo storage, as well as samba and syngthing are running.