I am a futurist. I tell stories about the future on-stage and online, and when I’m not telling them I try to write them, in non-fiction and in fiction, and it’s taken me nearly thirty years to understand why.

To be clear, I’m not talking about reasons to write, of which just one – the art and beauty of it – is sufficient. Hell, if I wasn’t writing the future, I would still be writing. And I’m not talking about the fun of it, the joy of speculating and playing with ideas, the stimulation (the future is never boring). No, I’m talking about the deeper reasons that keep me returning to a genre where to write ‘too soon’ is to say nothing and to write ‘too far’ is to risk irrelevance, where we risk being blindsided every day and where every foolish prediction can be called to account forever while our best predictions, the worthwhile ones, can attract dissent, disagreement, downright hate – or worse, be ignored. I’m talking about the reasons that carry me through the many, many shitty moments when the work feels immense and the results so pitiful I rip paper and scream, ‘To hell with it!’

So, why? Why persist with such a perilous undertaking?

CONCEIT – excessive pride in oneself; excessive appreciation of one’s own worth or virtue.

Here’s where it starts: Thirty years ago, I’m in a coffeeshop, staring dumbly at a journalist and her editor who fixes me with a no-nonsense stare and asks again, would I like to write a column in their magazine? Say what? What good am I as a writer? I have no credentials beyond a job that involves researching technology trends, no portfolio beyond the dry reports I hack together for clients. How on earth can I produce (gasp) eight hundred fresh words a month! Of course I say none of this. I say yes. Why? Because I’m inflated by some imagined aptitude for writing what the future could be, because I think I have something to say, and because – worse, much worse – I think I might be able to make a difference. Oh I don’t mean peace on earth and converting Brazilian presidents into tree-huggers, nothing so ambitious. More like a back-of-the-brain notion that maybe, just maybe, I could nudge an investor or two away from coal? Or how about inspiring a few students to get into renewables? Or awakening a few cops to better ways of doing community engagement? Or guilting a few mums and dads into throttling the firehoses of digital toxicity spewing into their homes? Maybe that level of ambition.

Conceit carried me through years while I groped for style and struggled for time, and my tone and topic choices lurched between the featherweight (faulty chips in Sidewinder missiles, anyone?) and what another journalist friend would later describe, with tepidly arched eyebrow, as ‘worthy’. Conceit pushed my pontifications into health care, policy-making, taxation and so many other areas where, according to convention, I had no right to offer any opinion at all.

The conceit is still with me, still greases the wheels. I’ve recalibrated downwards, though. Reality does that. It’s the scientists who discover, philanthropists who spend, entrepreneurs who invent, activists who block – these are the people who change the world. Writers can influence or amplify or maybe plant a seed or two, but unless we are very great, the changes incited by our laboured scribblings are small. Still, the truth is, every time I open my laptop or walk on a stage, I believe (hope?) I can wrench one heart, prick one conscience, open one mind. It’s my way of helping the world be a better place. Otherwise why do it? Why would any of us do it? It’s possible, netting up the sum of my work, that I alter nothing. But I don’t believe it. I can’t.

FREEDOM – the power or right to act, speak, or think as one wants.

Scene: Night. A long drive from Sydney to Melbourne. I check into a cheap small-town motel, open my laptop and begin typing my first words of fiction. Tentative. An experiment. I have the barest whiff of a premise and no inkling of where it will go, but I’m irritated by a recent demonstration of lie-detection software being installed in call centres and I know I want to write about that.

Surprisingly, words flow. Slowly, then steadily, then more rapidly. The near-future. Chinatown, San Francisco. There is an explosion. A brothel. I sketch characters, give them lines, place them beside rubble piles and parachute them into police cars. There is the police surveillance specialist, Shari, who is Iranian-American and takes her job very seriously and there is more to her story but that’s for later. There is her dispatcher friend, Yolanda, working in the next room, who happens to know where Shari’s boyfriend is. There are bloodied casualties and attending paramedics and flames and dust – so much dust! – and fragments of masonry peppering out of the sky like hail. And waiting in the wings an FBI agent and the cop who hates him and the Silicon Valley porn billionaire behind the brothels at loggerheads with the charismatic leader of the New Christian Organization of America … I’m laughing, actually laughing as I type. Did I experience this unshackling when I hacked out my first column? I guess I did. A version of it. But this is supercharged! Here, there are no facts to be checked, no sources to be quoted, no sub-editors waiting to blunt my message with a mismatched headline, and no present or historical constraints on the story, just my ideas spewing helter-skelter down the page. And when I finally stop and see the rudiments of Chapter 1, it feels like a whole new way of living. I remember telling myself, I’ll never write non-fiction again (I lied). I remember thinking, this was the way to say something. This was freedom. Except inside my skull it sounded more like FREEEEEDOM!!! And that, I thought, must explain everything.

It didn’t.

Scene: three years after the novel is published. Three years plus three hundred thousand words of drafting and plotting and tantrums and crying and rewrites and replotting and trying to salvage another story that works, I walk into my yard and get down on my knees and hammer a black-painted cross bearing the title of my second novel into the lawn, after which I snap a photo and send it to all my writer friends. It’s over. All of it. I can’t do this. Never again.

Second novel syndrome. So common. So pathetic. Whoever said it would be easy? Get back on the horse, Bruce! Except it really did feel like I had fallen off too many times, used up all the courage in the bucket and then burned the bucket too. Somewhere after the heroin-highs of that first book, Skinjob, hitting the bookshelves and the writer’s festivals and book readings, I lost my way.

There were signposts, had I bothered to read them. Such as the time a woman in a bookshop held the first novel up to my face and with a fierce look said: ‘I made my husband read this and I made my son read this.’ This was the fuel. She heard what I was trying to say and it made her angry.

Had I thought for a minute about why such moments felt so good, it might have saved me years of pain. But I didn’t. Instead I threw myself again and again into attempts to craft the ripping yarn, the structure, the vivid characters and so on. Four weeks after photographing that cross, I started a third novel. The premise seemed strong. The opening was fine. The story solid enough. It just took itself somewhere where I stopped caring and fifty thousand words later I gave that one up as a pile of shit too. Then I started a fourth …

SUBVERSION – seeking to destroy or undermine an established system or institution.

This next part is going to sound stupid. The beginnings of a solution came via a chance reading of a plaque in an art gallery. The plaque was placed there by a dead benefactor because he wanted to share a lesson a dead painter had once shared with him, and the lesson, paraphrased, was this: the best art is always, in some way, subversive.

I wish I could say it slapped me in the face, but it was more of a slow tap on my forehead. I turned the word over as I looked at other paintings. I liked it. I looked up the definition on my phone. I liked it more. I thought about books I admired. Fahrenheit 451? A Handmaid’s Tale? Everything written by Orwell? A Clockwork Orange? Subversive all of them, in character, language and message. The more I thought about it the more I realised all the movies and paintings and poems that ever stuck with me and refused to let go were subversive in some way.

And what of the digs at sexuality and religion and surveillance in Skinjob? Was this the true source of my joy there, too? Yes! Call it rebelliousness, contrarian-ness. Or, in Australian-speak, being a shit-stirrer. Jarring others to think differently is definitely, definitely, what drives me. I write the future to subvert. It’s who I am.

I should have photographed that damned plaque.

Understanding this helped me take a few of those fallen stories out of the trash and find the original spark that started them chuffing along in the first place, before they derailed. I’m reviving one now, and so far it’s staying on track because I know now what started the engine. When I’m ready, when I’m a little braver, I may even – deep breath – exhume the beast buried under that cross.

This new understanding also helped my public speaking which, in preparation at least, is also writing – if you count a man muttering at a page and looping long swirls around names, concepts, dates, words with coloured pens as writing. Certainly, it is storytelling. I used to start climate talks with science. Now, when addressing an audience predisposed to shut their ears at the first mention of biodiversity loss or submerged islands, I rabbit-punch them with, ‘There is more new wealth to be created from the transition to net-zero than any other human endeavour in history!’ It’s one of the most effective openings I’ve written. And it’s true.

Sometimes, when I’m presenting on the future of medicine to medical professionals, in between the wonder-stories of bioprinted organs and longevity molecules and cancer cures, I’ll slip in a story about a single homeless person, and use it to challenge assumptions about what can be done to make US health care affordable. That story, poorly told, carries the risk of being interpreted as an attack on their free-market values. And yet it is right and true, and to leave that pretty stiletto on the shelf instead of sliding it oh-so-delicately between their ribs, would not be who I am. So, I don’t. And happily, they seem to like it.

LOVE – an intense feeling of deep affection.

Honestly, if I stretch this story to its limits, this is where we hit bedrock, and it’s taken me a long time to dig through all the conceit and freedom-seeking and subversiveness to get there. Once, if asked to name an underlying emotion when taking myself off to write, I would have said irritation, even anger at where the world was going. What homo sapiens should be versus what we are. Either I was wrong, or I grew up. Now – and fair warning, this is going to sound cheesy – when I get out of bed at 3 AM to polish a presentation, the dominant emotion I tap into is … love. Big word, that. I wrote ‘care’ first, but care doesn’t cut it. So, love. Love for who they are and who they could be. And when I work on climate, when my blood is boiled by meetings with scientists who cry at the unchecked deforestation and emissions and the ocean sinks being filled and privately tell me we are destined to blow through two degrees no matter what and probably a lot more than two degrees because much of that CO2 stays up there hundreds of years and some of it thousands, when I’m white-hot about bullshit greenwashing technologies like atmospheric carbon capture because we will never, ever, have the stupendous quantities of excess renewable energy necessary to ‘vacuum clean the sky’, the emotion beneath the anger, the one that truly drives me to write yet another blog that may or may not amplify a message and might or might not inspire some small action, is love.

And here’s the weirdest part: the more I drown in our collected flaws and evils and stupidities, the more affection I feel for the whole sorry lot of us.

Name your irritant – crypto fraudsters, wealth inequality, AI deepfakes machine-gunning democracies – isn’t it love, deep down, that makes you want to shake the world by the shoulders and scream your message about the future? Say ‘futurist’ and what names come to mind? For me, Shelley, Huxley, Orwell, Alvin and Heidi Toffler, Atwood, Le Guin, Clarke, Sagan. What fuelled their kiss-in-the-dark awakenings and blunt-axe warnings? I see love. What about the works of David Suzuki or James Gleick or Simon Singh or Yuval Noah Harari? Love. What about the blackest of black futures in Neville Shute’s The Beach or Cormac McCarthy’s The Road? Love infuses every page.

COMMUNITY – the condition of sharing or having certain attitudes and interests in common

Picture this: I’m visiting the International Thermonuclear Experimental Reactor, the first full-scale fusion power plant being built in the world. I’m standing at the centre of the machine, peering into the guts of the atom smasher ringed by the strongest magnets ever created. The majesty! The Brobdingnagian scale of the thing! But most of all, the joy, because all around me toiling side-by-side to get us closer to a clean, green, unlimited-fuelled energy future are scientists and engineers from Europe, the US, Russia, China, South Korea, India and Japan. I turn to the director of construction, a Dutchman blessed with dry wit and infinite patience, and say it must be wonderful coming to work knowing he is dedicating every hour of every day to saving the planet. He deadpans, ‘No pressure then!’ We laugh. One of the happiest days of my life.

Company, kinship, fellow-travellers – shall we call it an extension of love? – I don’t know where this fits exactly, but it does, and it’s another motivation that took far too long to uncover. Or maybe I just needed to grow up.

Telling and writing the future connects me to my heroes. It sits me across the desk from a gerontologist trying to help us live longer, it lets me ask silly questions of a biologist trying to feed the planet, it affords me the privilege of listening as a roboticist shares his most far-reaching dreams. These people humble me and energise me. They have grown my purpose and evolved my methods; nowadays I start my stories with their frankly-spoken explanations, their aspirations and fears. They enrich me every single day. What is wealth? For those of us lucky enough to enjoy a roof and food and some modicum of security, the answer is not money – it’s relationships. As a wise lady once told me, some people are so poor all they have is money. To spend even a minute with these deeply good people who care about so much more than themselves, is a privilege beyond measure. And the same holds for every writer I’ve met through this journey, and every other artist, and every reader who has taken the time to reach out, and all the generous people who come up to discuss ideas after my presentations. I write the future to meet people who care about it.

I’m typing this essay in a hotel room with the window opened to the Manhattan sirens. Tomorrow, I’m meeting a philosopher to exchange ideas on where AI will take us. The next day, I’m presenting to a pharmaceutical firm. Then I’ll circle back to writing, punch out a few more pages of the novel. Soon after, I’m recording, because I’ve started a new folly – podcasting. I can’t spare the time because there’s twenty hours of weekend reading waiting, just to keep up on the science. The chances of making a buck out of it are nil. But it’s experimental, raw, conversational, and dare I say – subversive. I do it with a friend and there is a ton of laughter and we’re up to episode twenty and going like a train because every time we switch on the mics we’re thinking of the wonderful people we’re reaching and the stories we’re about to share. Insanity? Folly? It feels that way often, but the feeling passes.

I am no great writer. The victories are small, the learning immense. Sometimes I hit a nerve and sometimes I don’t. But I’m stuck with it, driven by it, addicted to it, haunted and enraged by it, and if I am so lucky, I’ll keep doing it. I have to. And that – as best and as truthfully as I can explain – is why I write the future.

Dr Bruce McCabe is a futurist. He has delivered more than 500 presentations on five continents. He wrote the techno-thriller Skinjob (Random House, 2014) was a weekly columnist in The Australian and has published several hundred research papers, op-eds and short opinion-pieces. He tells weekly stories about the future with co-host Paul Jones on their podcast, Is the World F*cked?