State of Project Segfault: May 2023
This month, we made a lot of backend improvements, configured psf.lt domains for most of our services, setup hourly snapshots for VMs on Soleil and more!
Soleil Levant electricity issues (again)
Here we go again, as foreshadowed by the feature image of last month's blogpost, Soleil is having electricity issues. The server is consuming around 300-350W on average and going by this, it isn't sustainable for MrLeRien to pay for it by himself.
We plan to decommission Soleil and move it to an old Gen3 Ryzen 5 rig that MrLeRien has, which has almost the same amount of CPU power as that ancient Hitachi server, be it with less power consumption. Update 30th May: MrLeRien has ordered 32x4 128 GB RAM and 960x4 SSDs (to be run in ZFS RAID5) for the machine. We will be moving to that machine the next month.
These changes however are really expensive (~600 EUR), and have been funded from MrLeRien's pocket. We encourage you to donate so we can sustain the project.
Our shortened domain psf.lt has been setup
psf.lt, our new short domain, has been setup for most of our public-facing services.
For some services, we couldn't setup a shortened domain due to technical limitations, and hence made a redirect.
For example: https://j.psf.lt/HelloWorldMeeting would redirect you to https://jitsi.projectsegfau.lt/HelloWorldMeeting
Another issue that exists with the shortened url is SSO on Gitea, which doesn't work as of now.
A list of the psf.lt URLs for our services can be found on this hedgedoc.
Server News
DNSSEC is completely enabled on psf.lt
Unlike OVH, the registrar for projectsegfau.lt, psf.lt's registrar (Gandi) allows us to set custom DNSSEC records.
Thanks to this, we have a completely functional DNSSEC setup on psf.lt
Page on Wiki to document our Security practices
We now have a Security page on the Wiki to document the security practices we take.
We hope this helps make Project Segfault more transparent.
Pubnix's configs are now public
You can view the pubnix's configs at git.projectsegfau.lt:ProjectSegfault/pubnix-configs
QEMU Guest Agent is now setup on all our Soleil VMs
To get more accurate stats in the Proxmox dashboard, we have setup the QEMU Guest Agent on all VMs on Soleil Levant.
Automated hourly snapshots for Soleil Levant
We now have automated hourly snapshots of all Project Segfault related VMs on Soleil Levant. This is yet another step in making Soleil Levant more resilient.
However, it currently doesn't work for DockerVM, DatabaseVM and CoreVM because their disks are in raw format, which isn't supported by proxmox's snapshots.
PS: We use https://github.com/ponceto/pve-autosnap for this
Iperf3 on all nodes
We now have an iperf3 instance on all our nodes, so you can check how fast your connection to our servers is.
Information on it can be found on the Iperf3 page on our Wiki.
Service News
Cal.com instance discontinued
We have discontinued the instance because calcom's docker builds have been broken for us since 2.6.12 (latest is 2.9.4) and we haven't been able to fix it.
It was also not actively used either, so yeah.
Future of Librarian
Librarian, the privacy frontend for LBRY/Odyssey, was recently archived by its developer.
Our instance however, will continue to be up, though without any maintanence.
We will move to a fork of the same if one pops up later on.
MyChecks/Healthchecks
We now have an instance of MyChecks/Healthchecks. This was created mainly for internal use to monitor our cronjobs, but you can use it as well!
It makes cronjob monitoring simple and efficient, and allows you to get notified when a cronjob goes down through email, matrix and any other service supported by the apprise module.
Soju (Pubnix)
The Pubnix now has an IRC bouncer in the form of soju.
It was mostly setup by Arya for him to use but any pubnix user can as well!
You can follow the IRC_Bouncer page on our Wiki to set it up.
Bifrost XMPP Bridge
We recently deployed aria-net's fork of the Bifrost XMPP bridge.
You can find instructions on how to use it at the wiki page.
-Arya