Tax policy is also used a method of encourage certain types of behavior and discourage others. EV Tax Credits and Solar Tax Credits are example of encouraging starting up industries which we need to assist with climate change in economic powered method.
At a broad level, offering benefits for marriage solves political problem, married people tend to vote so need to be catered to. It also solves societal one, marriage tends to be better at extremely broad strokes for society. Married Couples live longer, commit less crime, kids in married households generally have better outcomes and so forth. So politicians in United States decided to incentive it.