I use k8s at work and have built a k8s cluster in my homelab… but I did not like it. I tore it down, and currently using podman, and don’t think I would go back to k8s (though I would definitely use docker as an alternative to podman and would probably even recommend it over podman for beginners even though I’ve settled on podman for myself).

1. K8s itself is quite resource-consuming, especially on ram. My homelab is built on old/junk hardware from retired workstations. I don’t want the kubelet itself sucking up half my ram. Things like k3s help with this considerably, but that’s not quite precisely k8s either. If I’m going to start trimming off the parts of k8s I don’t need, I end up going all the way to single-node podman/docker… not the halfway point that is k3s.
2. If you don’t use hostNetworking, the k8s model of traffic routes only with the cluster except for egress is all pure overhead. It’s totally necessary with you have a thousand engineers slinging services around your cluster, but there’s no benefit to this level fo rigor in service management in a homelab. Here again, the networking in podman/docker is more straightforward and maps better to the stuff I want to do in my homelab.
3. Podman accepts a subset of k8s resource-yaml as a docker-compose-like config interface. This lets me use my familiarity with k8s configs iny podman setup.

Overall, the simplicity and lightweight resource consumption of podman/docker are are what I value at home. The extra layers of abstraction and constraints k8s employs are valuable at work, where we have a lot of machines and alot of people that must coordinate effectively… but I don’t have those problems at home and the overhead (compute overhead, conceptual overhead, and config-overhesd) of k8s’ solutions to them is annoying there.

The Beehaw admins made this choice, and documented their rationale here: https://beehaw.org/post/567170

This is a great approach, but I find myself not trusting Jellyfin’s preauth security posture. I’m just too concerned about a remote unauthenticated exploit that 2fa does nothing to prevent.

As a result, I’m much happier having Jellyfin access gated behind tailscale or something similar, at which point brute force attacks against Jellyfin directly become impossible in normal operation and I don’t sweat 2fa much anymore. This is also 100% client compatible as tailscale is transparent to the client, and also protects against brute force vs Jellyfin as direct network communication with Jellyfin isn’t possible. And of course, Tailscale has a very tightly controlled preauth attack surface… essentially none of you use the free/commercial tailscale and even self-hosting headscale I’m much more inclined to trust their code as being security-concscious than Jellyfin’s.

… advertisement and push they did on sites like reddit…

The lemmy world admins advertised on Reddit? Can you link an example?

… their listing on join-lemmy.org…

Until recently EVERY lemmy instance was listed on join-lemmy.

And with the name Lemmy.world they did nothing to dissuade anyone from thinking that.

They run a family of servers under the world tld, including at least mastodon, lemmy, and calckey. They’re all named similarly.

I also saw nothing from .world not claiming to be the bigger instance(super lemmy)

They ARE the biggest instance, but that happened organically. It’s not based on any marketing claims from the admin team about being a flagship/super/mega/whatever instance. People just joined, and the admins didn’t stop them (nor should they). It’s not a conspiracy to take over lemmy. It’s just an instance that… until recently… happened to work pretty well when some were struggling.

I think the issue is that .world has put itself forward as some sort of super lemmy.

Citation needed. All the admins of lemmy world ever purported to do was host a well-run general-purpose (aka not topic-oriented) lemmy instance. It was and remains that, and part of being a well-run general purpose instance is managing legal risk when a small subset of the community generates an outsized portion of it.

Being well run meant that they scaled up and remained operational during the first reddit migration wave. People appreciated that, but continuing to function does not amount to a declaration of being a super lemmy.

World also has kept signups open through good times, and more recently bad. Other instances at various times shut down signups or put irritating steps and purity tests along the way. Keeping signups open is a pretty bare-minimum bar for running a service though, it is again not a declaration of being a super-lemmy.

Essentially lemmy world just… kept working (until recently when it has done a pretty poor job of that). I dunno where you found a declaration that lemmy world is a super-lemmy, but it’s not coming from the lemmy world admins, it’s likely randos spouting off.

OP is claiming that they agree with lemmy world’s defederation choices driven by CSAM, which is unquestionably nonsense. Lemmy world admins have made several in depth posts explaining defederation decisions and none of them had anything to do with CSAM. In some jurisdictions, it would likely be illegal to give such an explanation as it would amount to creating a pointer to a source of CSAM that hasn’t yet been taken down. By and large, these things are reported directly to law enforcement and cleaned up quietly, without showing up in modlogs… and in many jurisdictions the law REQUIRES handling CSAM in precisely that fashion in order to prevent it from being archived before it’s taken down.

Is there a non-zero amount of CSAM in the Fediverse? Sadly yes. Once you achieve a certain scale, people do all the things… even the bad ones. This research paper (from Stanford, it’s reputable and doesn’t include or link to CSAM) discusses finding, in a sample of 320k Mastodon posts, over 100 verified samples of CSAM and something like 1k-3k likely adjacent posts (for example that use associated keywords). It’s pretty likely that somewhere on Lemmy there are a non-zero number of such posts, unfortunately. But moderators of all major instances are committed to taking appropriate steps to respond and prevent reoccurrence.

Additionally, blahaj.zone defederated from lemmynsfw over the adorableporn community. The lemmynsfw admins take reports of CSAM very seriously, and the blahaj admins stopped short of accusing them of hosting actual CSAM. But they claimed that models of verified age “looked too young” and that the community was courting pederasts. These claims were largely baseless, but there was a scuffle and some of the secondary and tertiary discussion threw around terms like CSAM loosely and incorrectly.

I think OP is probably hearing echoes of these kinds of discussions 3rd hand and just not paying attention to details. There’s certainly no well-known and widely federated CSAM communities, and all responsible admins would take immediate action if anything like that was found. CSAM doesn’t factor into public federation decisions, because sources of CSAM can’t be discussed publicly. Responding to it is part of moderation at scale though, and somewhere some lemmy admin has probably had to do so.

I think a couple things are in play:

• Very few people consumed these comics as we are… reading each one in sequence. You’d more likely sporadically encounter them in the funnies section of a physical newspaper. Which was a pretty hit/miss proposition to begin with. No one expected every one to be a winner, and people would routinely skip over stuff that didn’t interest them without thinking about it too hard. You’re operating under the assumption that Far Side is a classic, but at the time people would just cruise by and think “that comic is stupid, just like 60% of the other stupid comics on this page”. And folks were pretty happy to have 40% of comics be a bit funny.
• What made Far Side a classic was not its consistency. Rather, there were a few strips that became cultural phenomena. Basically a handful of hits that were breakout memes of the 80s and 90s. Colleges used to sell t-shirts of the school for the gifted strip with the kid pushing on the door that says pull, which is pretty accessible and one of those breakout hits.
• Because of those breakout hit strips, some folks got into Larson’s style of humor enough that fewer of his strips were inscrutable to them and he had a lasting market.
• Other comments point about topical references and those are also a big deal. If someone sees a beans meme with no context 30y from now, it ain’t gonna be funny. But a few weeks ago on lemmy, it was part of a contextual zeitgeist that was more or less about “these idiots will upvote anything, I’m one of the idiots… I’ll upvote this!” and it kind of captured the exuberant excitement of not knowing what lemmy was but wanting it to be something. Similarly, these strips often weren’t intended to last multiple generations. They assumed you were reading the newspaper RIGHT NOW… and so could reference current events very obliquely and still be accessible.

TLDR: Like a stupid meme, many Larson comics require shared transient context we’re missing now. Some are also just fukin weird, like cow tools. But some were very accessible and became hugely popular. These mega-star strips cemented Far Side’s popularity, and which gave Larson the autonomy to stay weird when he chose. Now we waste time trying to figure out what they meant.

This, but desktop linux users are on the step for 193rd place while excitedly screaming and holding a third-place sign. Steamdeck users are on the 3rd-place step while calmly playing their deck.

Captain’s log, stardate 41153.7: We’ve left planet Lemmy 17. It was… too communist or… not communist enough. I… can’t remember which. The search continues for a… planet… whose communism… is just right.

Goldilocks, take us deeper into the Fediverse. It must… be out there somewhere.

You haven’t stated your geographic location… but the common paid/legitimate approach is https://f1tv.formula1.com/ if it’s available in your area. If you are a sky subscriber, I think they have an online thing too.

Mods are working on community rules, and starting next week discussion of pirate sources will be against community rules. For now they’re taking no action on piracy threads and than to make an advisory comment similar to this one (I’m not a mod though), as they haven’t finalized/published the rules. But non-legit sources won’t be kosher to discuss next week.

You should post this as an end-user workaround in https://github.com/LemmyNet/lemmy/issues/506. LOTS of people are interested in this, and they commonly get redirected to that ticket to watch progress. If there was a workaround that didn’t depend on the devs available, lots of semi-technical folks could help themselves before the feature is done. It could maybe even help someone understand how to get started on the ticket to see what API calls are involved.

You can’t use your lemmy.world login to login to pathfinder.social, but you don’t have to. You can “subscribe” to communities hosted on pathfinder social, and federation will copy the the posts/comments to lemmy.world for you to read and vote on. Your own comments will get copied in the other direction as well for others to see.

You and I randomly happen to have accounts on the same lemmy instance, and I’ve already subbed to a bunch of the p2fe communities, check out this set of community search results and just click subscribe on whatever interests you.

To find other fun communities to subscribe to, check out the list of communities other lemmy.world users have already subbed. There’s ones for rpg, fate, shadowrun as well. Then if you’re still looking for new communities to sub, check out https://lemmy.blahaj.zone/post/61827 which explains the unfortunately confusing process of discovering new remote communities and teaching lemmy.world about them. It’s a bit weird and involves searching in multiple places and then doing a bit of a dance to set it all up, but it’s not THAT bad once you get the hang of it. Just confusing if no one teaches you the steps.

Good luck, have fun.

This may be obvious and not what you want to do, but Lemmy’s own UI is optimized for threaded conversations within a post. Browsing Lemmy communities from an account on a Lemmy instance looks pretty sensible.

That doesn’t help you manage comment toots from Mastodon, but if you don’t find another solution… it will look better from Lemmy itself.

There are other reports of this behavior as well: https://lemmy.ml/post/1159652

If cockroach is truly PG compatible, lemmy admins can swap it in without developer support. I suspect Cockroach constrains some SQL features and has poor performance on others, but that or AWS Aurora are things you can experiment with without dev support if you’re passionate about the proving out the value of scale-out.

The statement that spawned my response though was this:

I think lemmy will be bitten in the ass by not having considered clustering/horizontal scaling from the start. Federation alone as a scaling mechanism is only feasible for “nerds”. But if the network wants to grow, we will need a few scale-able large hosted instances.

I still don’t think it’s true that we need horizontal scaling to support sufficiently large instances. The amount of vertical and horizontal scaling ability built into Lemmy today is both useful, and likely to outstrip the current ability of its code to scale a single instance. Any algorithms that scale super-linearly with respect to comment-count, post-count, user-count, or community-count, will fail just as hard with distributed backends as they do with an RDBMS. And as you note, PG-compatible distributed systems provide a potential lower-engineering-cost on-ramp to distributed systems once the codebase is efficient-enough to warrant such a transition to scale further. I suspect I’ve contributed everything of use I have to this thread though, and don’t expect to respond further.

I think the communities are sorted by the number of subscribers.

It’s definitely not a strict sort by subscriber count. The top-5 communities in order from 1 to 5 have: 2491, 4461, 1414, 1138, 7109. It is sort of roughly ordered by some measure of popularity, but not one I can follow, the second entry is more popular than the first in every metric shown.

Don’t get me wrong, it’s a super useful service and one I use and recommend. But I was wishing for more sortability when I was setting up my subscriptions the other day, and if the number of communities grows that will be even more useful.

Doesn’t solve the availability issues, though. I know of no seriously hosted system that doesn’t have at least two replicas in different availability zones.

I’m not sure why you think the setup I’ve described can’t have coverage in multiple availability zones. If the lemmy and lemmy-ui containers are stateless as I suspect, you can autoscale them. Pictrs is new to me, not sure there… but it appears to support object-storage which would likely make it stateless and the object-storage can replicate to multiple-az’s. Postgres read-replicas can be placed in multiple az’s as well. The only component that presents an issue is the Postgres write-leader, and failovers there can be done in minutes. Many many popular sites run with an infrastructure like this and achieve excellent uptimes.

I do get the power of horizontal scalability, I specialize in distributed databases. But they come at a cost in flexibility relative to something like Postgres… and we’re very far from “needing” horizontally scaling database writes here. Everything else looks like it can be scaled horizontally if someone wants to take on the headache of doing so.

I’m familiar with that, does it have the sorting options that OP and I are discussing? It looks to me like the sort order is both hard-coded and confusing, although less-popular communities do seem to be at the bottom… I can’t identify any specific order that entries are listed in.

Yeah, it would be really nice to be able to look at the “popular” communities by sorting by most-subscribed, most posts, and most comments.

I think you probably underestimate how far one can get with “vertical” scaling. Here’s the dockerfile: https://raw.githubusercontent.com/LemmyNet/lemmy/release/v0.17/docker/prod/docker-compose.yml

• It includes 4 different containers… so there’s a way to scale out to 4 machines right away. Maybe not every container is doing an equal amount of work… but there’s some amount of immediately available machine-splitting.
• I’m no expert, but I believe that at least the lemmy and lemmy-ui containers are stateless. If so, they’re horizontally scalable already.
• Postgres then would likely be the main bottleneck. But postgres offers read-replicas, so again the write-load and the read-load can be hosted on separate machines. And if there’s enough read-load, you can have many replicas.

Other comments from the admins have shown that lemmy.ml today is running on a single eight-core box and it’s currently hosting 30k registered users and over 1k active. So how much more compute capacity can we throw at “vertical” scaling on the current software architecture?

• Just by going to a bigger single box, we can get 128 cores with no problem, a 16x bump in capacity. Does that get us to at least to 300k registered + 10k active?
• Splitting the containers onto 4 separate machines. Does that get us 2x more?
• Adding PG read-replicas and additional lemmy/lemm-ui containers would allow us to expand our instance footprint to maybe 6 physical machines should get us another 2x or more in performance.

Conservatively, that’s 100x the computing capacity of the current hardware and could potentially support 1m registered users and 50k active. Now, I don’t REALLY expect this to be possible today, there will be many software bottlenecks found along the way to scaling a single instance this large. But my point is that there’s already a medium amount of horizontal scalability built into lemmy, and if the software doesn’t fall over for algorithmic reasons (which is will at first), the current infrastructure architecture allows quite a lot of growth. There’s plenty of time between now and a federation of million user instances to adopt a truly distributed storage backend if needed.