Natasha told Q Magazine:

Every time I write an album, the first track is my favourite in a way. This one especially because the content of the song is a really good manifesto for the rest of the record. It starts out haunted by being lonely or not able to get inspiration, feeling a bit isolated. By the end it moves into this pro-life, elated full-on ending. For me, the album is about letting go of things and resolving something. I think because I wrote it over such a long time – a period of two years – I was almost trying to get away from extremes, trying to be a real person and settle down and find out about those bits in between the drama of feeling really sad and fucked up and really euphoric and happy, because I’d written the last album in that dramatic dark place. I wanted the The Haunted Man to be a bit more eclectic and varied and rich, I suppose.

In a Q&A session in New York, she explained what drew influence to write the song:

I was at home on my sofa with the autoharp in my lap and I was watching a film called Ryan’s Daughter which is a old 70’s David Lean film. He’s an English director and he made this film set in an island on the coast. And it’s a very romantic story of a woman who has an affair with a soldier that’s come back from war. I watched the film and there’s a bit where she runs outside and all these lilies are swaying really fast in the wind and all the pollen hits her skirt. And she’s looking up the hill and she sees the shadow of the man she loves. And then it’s this triumphum moment so I put the mute button on and I just wrote ‘Lilies’ there in about 20 minutes.