> counted my house as a tree
Even the more sophisticated algorithms pretty much always do this ;-)
You are probably not interested in taking this further, but you could give the Li tree filter a try: https://pdal.org/en/stable/stages/filters.litree.html
But getting perfect segmentation is basically impossible.