One overlooked thing is that there will be increased benefits in social programs (more tax theft), funny how the so called anarchists never mention this.
This is an incredibly simplistic take on a very complicated subject, and unfortunately, there's no short and simple way to explain why it makes no sense. I brush the surface of it here and thoroughly debunk this fallacious view, but you probably won't read it. The misconception is tied to an oversimplification of the myriad ways that government intrudes into our lives and the way that intrusion is affected by contracts between individuals.
The short answer is that for government to intervene and ban what should be a personal contract between consenting adults is an intrusion of government. Libertarians are big on consensual contracts. If they stop banning contracts between consenting adults, that's a retraction of government size and intrusiveness and clearly a step in a more libertarian direction. The phrase "get government out of marriage" is tossed around flippantly without thinking about all that would entail and what it will really take to get there (practically an end to most if not all of what governments do-- taxation, border control, the entire welfare state), as much as I want to get there too.
Actually, a number of individuals have made that point, as it is yet one more reason to support gay marriage: those who are getting "assistance" will get less as a married couple than as two separate individuals, so it further reduces the thieving...
It usually increases taxes for most because for most couples, both partners work. And since that's the case for most modern couples (and likely at least as much so for same-sex couples), you're right. It would generally mean they'd pay more taxes than singles.