Prestige: Simple, maybe, but not easy.

The concept of prestige in social learning can be difficult to address, and even more-so in the context of scientific practice. But a rumination of The Prestige as presented in Christopher Nolan’s film brings some resolution to conceptual tensions and inspires some real magic worthy of the term.

My friend Lotty (Charlotte Brand) wrote a paper about prestige (and I helped a bit).

Prestige is an interesting concept to me as a scientist. It’s not just an abstract idea being studied from an objective distance. It’s a characteristic relevant to the life and career of a practicing scientist. On one hand there is an ideal view of a prestigious scientist, respected for their trustworthy intellect. On the other, there is prestige bestowed on those with power, publication impact and profitability.

I spent many hours sorting conceptual tangles of what prestige means in the context of social learning during the writing of this paper, alongside what it means to the individual scientist. I’m still not done, but in writing this article I thought about a different kind of prestige. In particular, Christopher Nolan’s The Prestige. Somehow that led me to find some small resolution to my conceptual tensions.

You see, every magic trick consists of three parts…

The Pledge

At first, there is something ordinary. This is the Pledge.

Imk3nnyma / CC BY-SA (https://creativecommons.org/licenses/by-sa/4.0)

This paper began with a pledge to test some very ordinary things. Specifically, Lotty pre-registered the experimental design including its attendant hypotheses and analytical approach. She did this because it is the essence of good scientific practice. But she also pre-registered in full knowledge that ordinary scientists would have skipped to a later stage of research. What is ordinary in scientific practice isn’t the ideal of ordinary science.

Ordinary science is mostly mundane, careful and rigorous. It takes the stuff of common nature, and subjects it to simple tests. This gives confidence that the tapestry of human ideas can hold its own weight. But science in practice is often more about scoring points in a game of arbitrary metrics like publication count and impact factor. With this as the focus, the literature we create is corrupted by nonsense born from systemic bias and bad individual practices like p-hacking and HARKing. As an outcome, the tapestry is all worn out and coming undone. To early career researchers like Lotty and I, this breeds despair.

For me, I was so disillusioned with academia that I left to practice science on my own terms as a freelancer. I didn’t want my pursuit of understanding to be dictated by arbitrary metrics. Funnily enough, most of my income now comes as a writing consultant to people who are forced to keep pace with publications if they want to keep their career. But I still get the chance to participate in research, as is the case here.

Lotty had a different response to her disillusionment. She recognized the power and potential to change the course of things. She has written about it eloquently and passionately here, and this article is really just an attempt to re-frame her argument. But in a nut-shell, what it comes down to is frequency-dependence: individual scientists can only benefit by being good scientists if enough people are doing it. To Lotty, and now to me, reshaping the norms of scientific practice begins with the choice to pre-register. Science, like a magic trick, begins with a Pledge.

For our paper, making that initial pledge at pre-registration gave us confidence to stand by our results. All the tests and their interpretations were laid out a priori, so the end-result can be trusted. We can also be more confident that we’re not unintentionally spreading nonsense, and feel satisfied that we’ve contributed (in a small and ordinary way) to building a more integral understanding of nature.

Pre-registration also gave us the confidence to look beyond our pledged intentions. Some of this paper’s most interesting bits are speculative and based on unanticipated observations. But unlike what many expect, pre-registration does not limit the scope of what a paper can be about. In reality, the pre-registration gave us license to be open, honest and transparent with our explorations into the realm of speculation. We were freed from analysis paralysis and the subsequent self-deception inherent to HARKing, and freed from the nagging feeling that we were polluting the scientific corpus. We had laid a solid step forward to the theory we were testing, but we also set up materials for the next iteration.

Really just some simple, ordinary, every-day science.

The Turn

The second part of a magic trick is the Turn. Here, the ordinary is hidden and replaced by the unknown.

Photo by Sheri Hooley on Unsplash

If we turn over our completely ordinary scientific paper, underneath is an unseen series of transformative and enlightening events.

From my side, this came from the development of novel experimental software. Most of my years as a post-doctoral researcher weren’t spent writing papers, but designing an experimental system for social learning research. The experiments in our paper involve the proof-of-concept for that system. But getting to this point took years of design, programming and debugging. I would even set up ghostly rituals in an empty departmental foyer during the depths of autumn midnight, playing silent robots against each other in a bug-filled game. All done in preparation for the main event: to apply the system to real people in a real experiment.

From Lotty’s side, she went through something similar to get the quiz component running through Dallinger. Here she was thrust into a high-stakes learning environment, often having to hunt down bugs in the code at the same time that participants filled her inbox with problems. Due to time zone differences that come with international collaboration she was sitting at the computer past her bedtime, watching the experiment go live and praying to the coding-gods that it all ran smooth. When something went wrong (as it always does), she would deal with angry emails from understandably frustrated participants requiring compensation. It was a tense and frantic time, but now she’s sharp, vigilant, and can respond to 36 emails in four minutes flat.

Together, we learned how to wield our developing skills to best approach the data. Lotty was mastering the arts of Statistical Rethinking, and I was coding the tools to derive variables of interest. We spent some of this time with me crashing at Lotty’s eccentric landlady’s house in Falmouth.

Collaborative learning is something we both value about science (and also the topic of our research). When we weren’t coding, we were talking ideas and plotting future studies in Beerwolf (a pub-come-bookshop). That convergence of minds refreshed and inspired us both in ways that continue to this day. Our collaboration and sharing of information opened whole new territories in our scientific journeys.

Working on this paper also gave us the opportunity to spend time as friends. One of the things I miss most about the scientific institution is its ability to connect people with similar passions. I’m grateful that this line of research led me to a better understanding of the world, but more importantly it led me to a friend. I will always remember this paper, not because of its scientific merit or the skills I honed, but because it’s the time my friend Lotty taught me how to surf.

This is what it looks like Behind the Paper.

The Prestige

The third part of a magic trick is the snap back from the hidden to the ordinary. The Prestige.

Photo by Indrajeet Choudhary on Unsplash

So to bring it back to ordinary science, we wrote a simple paper about the concept of prestige as it relates to social learning. But behind this paper lies a dedication to scientific values and collaborative learning that developed us both into better scientists. Unfortunately, we still can’t shake the feeling that little of this experience helps us survive as scientists.

Rigorous, simple and ordinary science takes years of work behind the scenes to be trustworthy. It can’t be pumped off an assembly line to hit quotas. But this is the reality of what a scientific career demands.

Scientists consider themselves above crass magical tricks. But we have all been pulling one on ourselves for a long time. We pretend that our incentive structures don’t compromise scientific ideals. We pretend that science is being done reliably, despite clear and present evidence to the contrary.

But now is the time to end this trick we play on ourselves with some Prestige: to return things how they should be. And to pull off some true Prestige, we’re going to need some real magic.

Real magic isn’t like a trick. Real magic is bringing an idea, a mere thought, into cold, hard, objective reality. People perform real magic all the time. Just think of money. Enough people have agreed to believe that money has value, therefore it has value. So, if enough scientists (and publishers, university boards, hiring committees, etc) agree to believe that reproducible and transparent science has value, it will become valuable.

Working on this paper taught me how good scientific practice is done. It gave me hope that academia can improve. It helped me glimpse a world where scientists are valued as trusted intellects rather than as commodities.

And it can all start with something as simple as pre-registration. After all, that is what the scientific process asks. Then, if enough people are paying attention, we can spark some real magic and have an institution that values trust before impact. So I still don’t know what prestige is, but I’ve learned we can all get some if we do magic like this.

Ta-Daa!

Strobridge Litho. Co., Cincinnati & New York Restoration by trialsanderrors and Morn / CC BY (https://creativecommons.org/licenses/by/2.0)

[This article was first published at https://socialsciences.nature.com/posts/prestige-simple-maybe-but-not-easy]