The magnetic Queen singer is still attracting fans, 24 years after his death – yet he was remarkably hard to fathom. Can a BBC documentary, and a Hollywood film starring Sacha Baron Cohen, cast any new light on the man behind the legend?

In November 1991, days before he died, Freddie Mercury met his manager to discuss how best to reveal to the world that he had Aids. Once they had agreed the wording of an announcement, the 45-year-old singer began to wonder how he would be remembered. "You can do what you want with my music," he said, "but don't make me boring."

He got his wish. Twenty-one years later, Mercury is still the rock star's rock star, exerting enough magnetism to reel in new fans who are too young to remember him firsthand. There is a Hollywood biopic starring Sacha Baron Cohen in the works, and a new BBC documentary, The Great Pretender, tells the story of the last five years of his life. Additionally, Mercury's final solo album, Barcelona – the semi-operatic outing he recorded with soprano Montserrat Caballé – has just been re-released.

In the BBC programme – its title taken from a 1987 solo single – Mercury emerges as anything but boring. Skittish, exasperating and endearing, yes, but never dull. "His hedonistic streak, which culminated in a notorious 39th-birthday bash in Munich, is documented: Mercury holds court as men in drag mill around and a male guest dances naked. That may sound relatively tame, but apparently the real debauchery was conducted out of view of the camera. Ex-manager John Reid remembered it as "the last days of Berlin, the last hurrah" – after which Mercury settled down into something approaching a sedate existence. Equally, though, we see him reduced to a bag of nerves at the idea of working with Caballé, whose voice he called "the best in the world".

There are also new interviews with Caballé, the surviving members of Queen and a host of friends, and all are agreed on one point: Mercury was a fastidious, generous man whose private life was utterly off limits to anyone outside the inner circle. For someone who sold in the neighbourhood of 150 million albums, he was remarkably hard to fathom.

His inscrutability was his trump card, maintains Rhys Thomas, director of The Great Pretender. "There were a lot of layers to him. He was elusive." He cites as an example Mercury's insouciance about his illness. "He told Queen about it, and never spoke to them about it again. A couple of days before he died, [band manager] Jim Beach went round his house.

He'd stopped taking medication and he'd gone blind – he was in a really bad way. [But] all Freddie wanted to talk about was his music. 

"[Guitarist] Brian May was about to release a solo single, and Jim was saying he should postpone it. But Freddie said: 'Tell him to release it. I might drop at any minute – what better publicity could he have?'" (May did put the record out, a fortnight after Mercury died. It reached No 6 in the chart.)

Then there was his mastery of stagecraft: the former Farrokh Bulsara was an intensely physical performer who made every inch of the stage his domain. In front of 70,000 fans, at stadiums across the world, the intrinsically shy singer lost all trace of self-consciousness.

Sacha Baron Cohen will have a lot to live up to: he may look like Mercury – even if seven inches taller than the 5ft 9in singer – but can he command an audience like Freddie? Thomas chuckles: "I spoke to him the other week and he's really excited about doing it. He's even trying to learn to sing like him, practising the songs." That shows gameness on Baron Cohen's part; Mercury's voice was a force of nature with the velocity of a hurricane. Interestingly, though, it wasn't his "real" voice, or so claims Caballé. On the telephone from Barcelona, she says: "He had a baritone voice. I told him one day, 'Let's do a small duet of baritone and soprano,' and he said, 'No, no, my fans only know me as a rock singer and they will not recognise my voice if I sing in baritone.' So I didn't conquer him to do that."

Yet he might have agreed eventually. Caballé believes that if Mercury had lived, he would have delved more deeply into the classical music that increasingly captivated him in his last years. "We talked about doing something together, more classical. We also talked about recording Phantom of the Opera, which he loved very much. It's a moving thing – I knew he was in bed and very weak, and I wanted to surprise him by recording Phantom.

"I phoned him and put the speaker near the phone and played it. He was very happy. He said, 'Thank you, Montsy, I wanted very much to hear it.' And that was my last time I talked to him."

Lyricist Tim Rice, who co-wrote two tracks on Barcelona, says nonetheless that Mercury would never have abandoned Queen, who hit No 1 with their album The Miracle in 1989 and again with Innuendo in February 1991. "He'd have continued to work with them, but we were also talking about doing a musical together. The tragedy," Rice says, "is that he could've become a great writer of theatrical pieces, or even grand opera. He was always wanting to learn more about music."

Mercury's longtime personal assistant, Peter Freestone, who now works as an Aids campaigner in Prague, concurs. Even when Mercury became ill, he lived for his music. "[Once he contracted HIV] he never, ever spent time talking to me, wondering where he got it, which country, which city. It was a fact, it was happening. He knew he was going to die, so why waste time regretting? But he did have a regret at the end – that he still had music in him."

The indulgent lifestyle, the house in Kensington furnished with Japanese antiques, the wealth – they were just fringe benefits. The consensus is that he would have reunited with Queen, while building a parallel classical career. The last four years of his life were marked by an unprecedented burst of creativity that included not just the Barcelona project and the two hit albums, but also the recording of several tracks for Made in Heaven, the album posthumously completed and released in 1995. "I don't think he had any unfulfilled ambitions," Freestone says. "He considered himself lucky to be able to do what he had done."

Barcelona was his one of his proudest moments – but what would he have thought of the re-released version, which replaces Mercury's original keyboard arrangements with an 80-piece orchestra? "Oh, he'd have been overwhelmed. He'd be gobsmacked. There are so many more genuine climaxes with the orchestra. The nearest I ever saw him to crying was when Montserrat came in and put the first vocal tracks down for the song Barcelona. There were tears in his eyes."

But why is there still so much interest? Is it, as Thomas says, his elusiveness? With Mercury, the curtain went up and came down again, leaving us none the wiser about his real life. He didn't live into the internet era, and so never had a chance to destroy his own mystique by tweeting and sharing. Twenty-first century rock stars are handmaidens to their fans, compelled to expose their lives to scrutiny, whereas, with Mercury, there was a kind of lordly absence of detail.

"If he was here now, they'd have wanted him to go on Dancing with the Stars and The X-Factor and you name it. He'd have hated that. People's lives are now open books – is there anything we don't know about Lady Gaga?" Freestone says. "But also, Queen's music is timeless. I was out last night and saw a Queen cover band, and there wasn't a single person in the audience who could have seen them live because they were too young. But every one of them was dancing and singing the words. He was enigmatic – everything he did raised questions he didn't answer."

Meanwhile, Mercury's estate keeps getting richer. The musical We Will Rock You, based on Queen's songs, celebrated its 10th anniversary this year and has productions playing around the world. And Queen themselves keep touring, with other singers standing in for Mercury. Ironically, says Freestone, Mercury himself might have been dismayed at Queen's continued existence. "He hated the idea of bands getting back together. Oh, no, no, no."