Pauper is the obvious answer. It’s unique in that every card in the format is Common rarity, so that mitigates price already. Check out MTGGoldfish’s Metagame page for Pauper, which lists competitively viable decks and their price. Not sure how up to date it is, but it’s a good starting point!

I recommend Commander for playing in a group, since it has lots of fun gameplay, there are plenty of powerful budget lists and basically all the cards ever printed (with very few exceptions) are legal, so you can play what you have and never have to worry about rotation. I recommend looking up some budget lists to start out with, as they are generally significantly more powerful than preconstructed decks (some exceptions apply, but those precons are no longer cheap), while not being all that much more expensive.

Alternatively, as others have suggested, Pauper is a fun format to play that is basically guaranteed to stay fairly cheap, due to all the legal cards being printed at common at some point.

I recommend looking through (budget) decklists for either format with your friends and then agree on which one looks more fun for your group.