It certainly can be..trying. And even lonely.
But. I think the honor of having people 'love me' as you say, makes it a priviledge, to serve them in that way.

Making my people happy, making them feel safe, whatever I can give, makes me happy. I wouldn't trade the position for anything.