THIS is how it goes, his grandson says. A man falls in love with a woman. He is an artist. She inspires him, becomes his muse. He paints her over and over again.

And then the inspiration is used up. He goes seeking another. The relationship is not as important as the paintings. “The man and the artist are one person in the case of Pablo Picasso,” his grandson says. “He put the art first.”

It is Wednesday morning in New York. Later today Olivier Widmaier Picasso will fly to Edinburgh for an appearance at the Edinburgh International Book Festival. Yesterday he went and sat in the Museum of Modern Art and watched the people coming to look at one of his grandfather’s most famous paintings, Les Demoiselles d'Avignon, painted more than a century ago but still holding a huge pull for art lovers.

Indeed, it is hard to think of another name in modern art that is more resonant than Picasso. Between 2017 and next year there will be 42 exhibitions mounted of his work. And that’s just in Europe. He is the benchmark for the art market. Quite frankly, Picasso is modern art.

But what did that mean for those who share his surname? Those who grew up in the shadow of his legend?

Widmaier Picasso, a TV producer in his day job, has written a book, Picasso An Intimate Portrait that attempts to answer those questions. In it, he talks about Picasso the artist, his political beliefs and his financial dealings. But at heart it is the story of Picasso and the women he loved and the families (plural) he created. In life as in art Picasso, his grandson suggests, was avant garde.

Living in the shadow of Picasso could be comic. When he was a boy, Widmaier Picasso says, his art teachers used to praise his “horrible” drawings to his class mates, just because he was Picasso’s grandson.

But much of the story, it has to be said, is tragic. There were costs to belonging to the tribe of Picasso and some struggled with the legacy.

Widmaier Picasso’s grandmother Marie-Therese was one of them. She met Picasso in 1927. He was married to Olga Khokhlova at the time. Marie-Therese became Picasso’s lover and muse. They had a daughter together, Maya, Widmaier Picasso’s mother.

Indeed, Picasso was all set to divorce Olga in Spain where divorce had been legalised, but Olga wanted to remain Madame Picasso. “In 1935 in high society in Paris to be a divorced woman was to be excluded. She didn’t care about the money.”

General Franco then came to power and abolished divorce in Spain so Picasso could not divorce Olga anyway. “He kept his artworks and his fortune. He kept Marie Therese. And then he met Dora Maar.”

Marie There and Dora were in competition, Widmaier Picasso says. “Marie Therese was the sensual inspiration and Dora Maar was the intellectual stimulation.”

Growing up, Widmaier Picasso never met his grandfather. His mother never visited because she feared being turned away at the gate of his estate. But over the years his grandson found it impossible to escape the legacy of his grandfather’s name. It was the same for all of those children who inherited the Picasso name.

“I would say it’s not easy to live next to Picasso,” he says. “You are just waiting for your cheque or a word or attention. Or you live without Picasso, which is not possible because at some point someone will know that you are the grandson or the granddaughter of someone special.

“You have 90 per cent of your life influenced by history, by his legacy, his fame, and 10 per cent you can save as your own life.”

And not everyone can live with that. In the end Widmaier Picasso’s own grandmother committed suicide, as did Picasso’s second wife Jacqueline and Widmaier Picasso’s cousin, Pablito.

“For my grandmother life after Picasso was not so exciting. She was not in need of anything. She was probably tired. The same for Jacqueline.”

His cousin Pablito’s situation was different. He was 24 when he died, shortly after Picasso himself passed away. He drank a bottle of bleach having been turned away at the gate of Picasso’s house in Mougins.

Pablito’s death was a result of a “kind of despair,” Widmaier Picasso suggests.

“It has been said he was on drugs, that he was depressed,” he points out. But, really, he wonders, if the problem was the fact that his cousin shared the same name as his grandfather. “It was a huge mistake, a huge mistake. You cannot be Pablo Picasso after Pablo Picasso. It is impossible.”

Picasso’s life is a story of destruction as much as creation then. “He was not the man he should have been or the grandfather he should have been,” Widmaier Picasso acknowledges. “But he had too much to do outside of the family.”

And that in the end is the reality. Picasso’s story is not just a story of a disparate family. It is the story of modern art.

“I feel a bit uncomfortable when I say ‘my grandfather.’” Widmaier Picasso admits. “I have to share him with the rest of the world.”

Olivier Widmaier Picasso appears at the Edinburgh International Book Festival alongside James Attlee tomorrow morning at 11am. His book Picasso An Intimate Portrait is published by Tate Publishing, priced £30.