If the community’s server does not track downvotes, then indeed, everyone just sees downvotes by users local to their home instance. Otherwise, downvotes work the same as upvotes - that is, somewhat reliably - unless your frontend or user instance hides them but that’s your problem.