Neighbors Dispute

Hello! We've been having an issue with our neighbors, and I'm coming here to see if there's anything that we can do to legally get them off our backs.

About two years ago, we managed to buy a house. Which is great! But our first time meeting our new next door neighbors, they brought up an "agreement" with the previous owners that they would go halfsies on removing a cluster of trees on the property line. Legally, I don't see how that is something that we, as the new owners, are responsible for, as it seems to just be a verbal agreement.

Fast forward and this summer they got a quote about removing the trees, and also got the property line drawn up. Out of the three trees they want to remove, only 1/4 of ONE of the trees is on our property. Yet, they still have us a $3,000 quote and requested half in order to remove the trees. We've told them no, but as the summer is drawing to a close, they are starting to borderline harass us (i.e. standing on our porch and aggressively banging on our front door hard enough to make our windows rattle and to set off our reactive dogs for 10+ minutes, asking us about it everytime they see us outside, etc).

Again, I don't really see how we can be on the hook for this. But it's starting to grate on my nerves, especially when we still have so many repairs to the house that are far more necessary than getting rid of these trees. We just had to replace our boiler last year, and one of our radiators split open during the winter, so even if we are, we just don't have the extra funds to send their way.

Location: Minnesota