Active User Growth

!science@mander.xyz, Science, 25 => 216, 638 posts (6 this week)
!linux@linux.community, @linux on Linux.Community, 15 => 53, 31 posts (10 this week)
!thiccmoe@ani.social, ThiccMoe, 114 => 181, 621 posts (8 this week)
!palaeoecology@mander.xyz, Palaeoecology, 63 => 111, 85 posts (1 this week)
!europe@feddit.org, Europe, 1723 => 1808, 1098 posts (79 this week)
!mentalhealthsupport@midwest.social, MentalHealthSupport, 29 => 56, 45 posts (2 this week)
!movies@hexbear.net, Movies & TV, 476 => 518, 6494 posts (27 this week)

Subscriber Growth

!morphmoe@ani.social, MorphMoe, 309 => 331, 321 posts (7 this week)
!lobsters@lemmy.bestiver.se, Lobste.rs, 32 => 39, 1208 posts (171 this week)

  • FrostyTrichs@walledgarden.xyzEnglish
    3·
    1 month ago

    Lemmy seems to be on the fritz lately. I’ve noticed lots of weird federation/sync issues, with even the big places like .world having problems recently. Very strange behavior across the board the last couple weeks.

    • Andrew@piefed.socialEnglish
      6·
      1 month ago

      There’s been a recurring federation delay between lemmy.ml and lemmy.world:
      https://grafana.lem.rocks/d/cdfzs0dwal3pca/federation-health-time-behind?orgId=1&var-instance=lemmy.ml&var-remote_software=lemmy&from=now-30d&to=now&var-remote_instance=lemmy.world&timezone=browser

      The LW admins are aware, but I don’t think they’ve yet nailed down the cause (it fixed itself yesterday, but has started again today). The same thing might be stopping the crawler from getting a full API response from them.

      I have a test instance that’s subscribed to a couple of communities, but is quiet enough to be able to monitor federation, and if you do that, you realise that there are activities that are meant to happen, and just flat-out don’t. Other instances can recover (if they receive a reply for a post they don’t know about because LW forgot to Announce it, then they can fetch it instead), but it’s all a bit flaky, yeah.

      • FrostyTrichs@walledgarden.xyzEnglish
        3·
        1 month ago

        I’m having similar issues with a number of instances. Users have reported verified federation delays between walledgarden and several other places including lemmy.ml, hexbear.net, feddit.org, vegantheoryclub.org, and possibly others. Before anyone says it- yes I know they don’t federate with everyone but walledgarden isn’t being blocked/defedded.

        Walledgarden seems to be able to retrieve the data from them, but WG communities stopped federating to them a few days to a week or more ago in many cases, and as recently as yesterday in others. I’ve been working through it with our host but so far we haven’t figured it out since there isn’t anything obvious causing the problem on our side. As far as I can tell replies and interactions from walledgarden still make it to those instances without issue.

        It’s very frustrating.

        • Andrew@piefed.socialEnglish
          4·
          1 month ago

          I mentioned the other day that requests to walledgarden error (curl says ‘transferred a partial file’). If another instance doesn’t get a good response for these requests, then after 3 days, it’ll mark the other instance as dead, and not federate out to it. Once this happens, it’s a pain, because traffic from walledgarden doesn’t make them realise that it’s actually alive, you have to wait until it tries again and gets a successful response.

          If you haven’t done so already, it might be an idea to use curl to query your site, and see what the response in nginx says.

          • FrostyTrichs@walledgarden.xyzEnglish
            3·
            1 month ago

            Update to this-

            After talking with my host some adjustments were made that should correct the curl error. I still don’t believe this has anything to do with the lost sync but it’s one less thing to worry about. Thanks for the information and help!

          • FrostyTrichs@walledgarden.xyzEnglish
            2·
            1 month ago

            I saw your comment but I was mainly concerned with getting the instance to show up on lemmyverse at the time, which I suspected was the connection being blocked by our host (it was). Coincidentally since getting listed lemmyverse has been having its own problems so I put it on the back burner.

            I’ll do some deeper digging into this and see what I can figure out. I don’t think it has anything to do with the communities falling out of sync since it’s never mattered before now and WG isn’t the sole instance being impacted, but it should work correctly either way.