Been fiddling with #Sidekiq today to try and improve the performance of my Mastodon instance.

Beginning to suspect #Postgresql is the weak link, but not sure what else I can do with that (I know about PGTune)

#MastoAdmin #Mastodon

Thinking about moving my instance to docker. In your experience, are there any downsides besides the migration itself?

#mastodon #mastoadmin #docker #selfhost #selfhosted #selfhosting #homelab

#MastoAdmins - if you have problems with the "Fetch All Replies" function in v4.4 and 4.5 of Mastodon, check your "pull queue" in Sidekiq.

The function was working for me for a while, but then I started getting a circular "loading" icon at the bottoms of threads, and the remote replies would never appear.

This may have been because I added some more relays recently.

After chatting with some other admins in the Matrix support channel, I checked my Sidekiq queues, and there were 1.2m entries 😱

I found this article https://thomas-leister.de/en/scaling-up-mastodon/, which mentions splitting up the queue pollers into different services and I saw the same symptoms, htop was showing sidekiq's 25 threads, but only 1 of them was actually busy, and using 100% CPU, where other CPUs were mostly idle.

My machine is pretty beefy - A VM on SSDs, 4 vCPUs (Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz) and 48gb RAM, so the bottleneck is the lack of proper automatic multithreading in Sidekiq?

In the end I followed Thomas' instructions, and created 2 extra system services that just serviced the Mastodon "pull" queue (1 was not enough).

9 days later, the queue is now at 0, and reading posts in Mastodon now seems to pull in the new replies again 🥳

I have now disabled one of the 1 pull queue services, in the hope that one for the pull queue is enough.

😅

#MastoAdmin

Alt...

Screenshot of my Sidekiq queue, showing 999,879 items in the pull queue!

Alt...

Graph showing the drop in pull queue items over the last week. Processing seems to be linear.

yeah no i love it when the S3 server crashes at night

*prioritizes "create automatic fail recovery" in her todo*

#mastoadmin #fediadmin #garage

So, I built a new instance and you can find me at @mookie@weredreaming.com.

That doesn't mean that https://lifeiswoohoo.net is going away, it's staying for a while -- but lifeiswoohoo.net will be the instance that goes away if I ever want to change domains.

We're Dreaming (https://weredreaming.com) is an attempt at having an instance with permanence. It is where I plan to write and post without having posts disappear when I move instances, because it's obvious I'll always want to be building and destroying instances.

I'm still figuring out what I'll be posting here and what I'll be posting at We're Dreaming. But the initial thought is that more personal posts will go on We're Dreaming along with my movie posts.

I have already followed those of you that I interact with on a regular basis, if you wouldn't mind following back that would be awesome.

I am keeping the We're Dreaming account as one where I approve followers.

#fediverse #fediadmin #snac #snac2 #mastodon #mastoadmin #home

Zu verschenken:

Ich habe hier noch eine ganz passable Domain rumfliegen, die sicherlich eine großartige Fediverse Instanz abgeben könnte: tacheles.social

Ich würde zwar gerne selber, aber eine offene Instanz würde sicherlich schnell mein Budget sprengen.

Wer ehrliches Interesse hat, mit der Domain hier etwas zu machen, möge mich anschreiben.

#domain #instance #newinstance #mastoadmin #zuverschenken

If you follow anyone from the Mastodon instance `oldfriends.live they seem to be having some major problems at the moment.

They're reaching out asking for help from anyone who feels like they're enough of a #PostgresQL expert to help them troubleshoot what sounds like major database issues.

https://oldfriends.live

#OldFriends #MastoAdmin #DBA

A #SysAdmin / #MastoAdmin Question for folks

Has anyone used mikrotik hardware in anger / in production?

I've had enough now of my Unifi gear causing problems, this is the second gateway I've purchased which exhibits these random lock-ups / crashes and I'm very much over it so am looking at alternatives and mikrotik continues to be one that looks sensible.

Just curious if anyone has much experience using it hands on?

Looks good! @fanfare #kubernetes #goingdark #mastoadmin

Alt...

a uptime page showing green and 100%

In the past few weeks there has been a bot farm hammering small Mastodon instances with requests for what looks like emoji-related assets and the hashtag API.

More specifically, it appears this bot farm has now come for my Mastodon instance, because the number of requests has doubled in the last 24 hours, with a lot of the requests being API requests for a single JSON file.

Even more specifically, looking at data provided by my CDN provider, a lot of the bots appear to be using IP addresses associated with Tencent Singapore.

#MastoAdmin #Mastodon

Alt...

Two tables displaying the top 10 URLs for "birdbox.party": the left table ranks URLs by total requests, with "https://birdbox.party/api/v2/instance" leading at 94,107 requests. The right table ranks URLs by total traffic, with "https://birdbox.party/packs/assets/compact-NMaGTNYN.json" generating the most traffic at 5.28 GB. The data highlights the most accessed and bandwidth-intensive endpoints.

Someone just donated to goingdark.social for the first time.

I'm not gonna lie. running this instance takes work. Infrastructure doesn't maintain itself, storage costs money, and keeping things running smoothly isn't always easy.

This made my whole week. Thank you @lor, seriously. 🙌

https://ko-fi.com/home/coffeeshop?txid=465d2b80-08f0-4883-99d3-96e4e7ff825f&mode=public&img=ogsomeoneboughtme

#FediAdmin #SmallWeb #IndieWeb #mastoadmin

Really enjoyed the "Move Slowly and Build Bridges" talk this afternoon by @rwg, and the panel including @chad, Jordan (tag if you know their handle), @Django, and @KarenKeiller!

Also great to finally meet @robyn, @Chigaze, and @murteza, and to see @vga256 again!

#yeg #fediverse #Mastodon #MastoAdmin #MoveSlowlyAndBuildBridges

Alt...

A photo of Robert Gehl speaking at a podium.

Alt...

A photo of a panel on stage with Robert Gehl, Chad Ohman (mstdn.ca), Jordan (MacEwan U), Django (cosocial.ca), and Karen Keiller (MacEwan U).

Had a sudden loss of space on the Thirring mastodon server. (Enough to completely fill the disk and crash it) Not sure why. We'd had about 100G clear as of last week. Anyone notice anything weird out there? #MastoAdmin

If any self hosting admin out there had tried their hand in integrating Mastodon and Authentik, gimme a holla, I need to connect to the hive mind (aka brainstorming) :)

Boosts appreciated

Update: apparently no one has ever done this 😅 thank you for your cooperation everyone but I believe it was boosted enough

#Mastodon #MastoAdmin #Fediverse

Updated my test mastodon instance, listodon.com, to v4.5.6 and updated TangerineUI to a nice lagoon theme for the default.

I had some slowness during this upgrade, I'm not sure why. At one point I thought it was completely stalled but it finally completed.

#MastoAdmin

Alt...

The Mastodon Web UI, using the blue Lagoon theme from TangerineUI.

We’re happy to announce that Mastodon 4.5 is now ready for prime time! Quote posts, async fetching of replies, server timeline visibility settings, more moderation tools, and lots and lots of other tweaks and improvements.

Announcement on our blog: https://blog.joinmastodon.org/2025/11/mastodon-4.5/

Full release notes and update instructions are available on our GitHub release page: https://github.com/mastodon/mastodon/releases/tag/v4.5.0

#mastoadmin