Dolly Parton Opens Up About Keeping Husband Carl Dean Out of the Spotlight: ‘It’s Just Not Who He Is’

David Becker/Getty

Dolly Parton has a mutual agreement with husband Carl Dean when it comes to stepping into the spotlight.

The country music legend, 74, has enjoyed a 54-year marriage with Dean, 78, but she recently told Entertainment Tonight that some people joke that he is imaginary since he rarely takes part in Parton's fame-filled career life.

"A lot of people have thought that through the years because he does not want to be in the spotlight at all. It's just not who he is," Parton said. "He's, like, a quiet, reserved person and he figured if he ever got out there in that, he'd never get a minute's peace, and he's right about that."

"I've always respected and appreciated that in him, and I've always tried to keep him out of the limelight as much as I can," she added. "He said: 'I didn't choose this world, I chose you, and you chose that world. But we can keep our lives separate and together.' And we do and we have."

In fact, Parton went on to say that their time apart has been a secret to their lasting love.

"Our joke about the fact, when people ask me why it's lasted this long, I say, 'Because I stay gone.' And there's a lot of truth in that — the fact that we're not in each other's faces all the time," she said. "But we do have a great respect and admiration for each other. We both have a great sense of humor. So we have a lot of fun."

RELATED: Dolly Parton Shares Origin Behind Her Biggest Hits — and What Keeps Her 54-Year Marriage Strong

Kevin Winter/Getty Dolly Parton

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

In September, Parton opened up to PEOPLE about her marriage, explaining that she and Dean still have date nights together.

"We do! My husband and I, we've been together 56 years," said Parton, who met the Nashville businessman when she was 18 before they tied the knot in 1966. "We still have our little times, like in the springtime when the first yellow daffodils come out. Even if there's still some snow around it, my husband always brings me a bouquet. And he'll usually write me a little poem. Which to me, that's priceless. That's like a date in itself."

Parton added that another secret to her successful marriage is that she's always been comfortable in her own skin. "And that's sexy," she said.

RELATED VIDEO: Dolly Parton Addresses Those Tattoo Rumors, Reveals the Moving Reason Behind Her ‘Tasteful’ Ink

Dolly Parton Addresses Those Tattoo Rumors, Reveals the Moving Reason Behind Her ‘Tasteful’ Ink

Dolly Parton tells PEOPLE exclusively about her “ribbons, bows, and butterflies”

RELATED GALLERY: Celebrity Secrets for a Happy Marriage

"You may not want to dress or look like me, you can take from that to be yourself. I'm a very passionate person in everything that I do, and I think passion has great sex appeal. I think people are drawn to that because it's a magic. It's an energy. And I've always been a great lover because I'm passionate. I just feel my sensuality, my sexuality, my passion, and I don't have to apologize for any of that," she said.

"I've never been offensive in any way, never tried to be. Or I've never tried to be anything other than just what I am," added Parton.