At long last, I’ve published some academic research. I’ve been doing my PhD part-time since the end of 2022, and throughout that period I’ve had the goal of seeing my name on a published paper.
I developed a methodology and wrote some Python scripts to automatically review over 2,500 corporate publications and answer one simple question:
- Are companies reporting ranges of uncertainty when they publish their carbon footprints?
To give some background, the reason I think uncertainty matters is that corporate carbon footprints are pretty terrible. It’s not because they’re trying to fool people, it’s just that the methods for measuring carbon footprints are still developing, and lots of the data companies use is about as accurate as a Bus Éireann real time arrivals board. This is compounded by the fact that by far the largest part of any carbon footprint is stuff outside of the direct control of the company. We call this “Scope 3”, and about 90% of Scope 3 emissions are in the supply chain. For example, a company manufacturing cars in the UK might use some energy putting them together, but the biggest source of emissions by a mile is the factory where the steel input was produced, steel which they bought from China. To make things more complicated, people often don’t know exactly where their products came from. Traceability gets lost as the materials pass through many “tiers” of the supply chain, which are simply some kind of middlemen, like importers, intermediate processors, or brokers.
All of these elements combine to make form a thick paste of uncertainty. Puzzlingly, you’ll never see an acknowledgement of uncertainty slapped on a billboard or on the back of a product you bought in the shop. “We think our carbon footprint is low.” “We may have reduced our carbon footprint last year by 50%, give or take 20%.” “We are more sustainable than our competitors based on some assumptions. If our assumptions turn out not to be accurate, forget about this claim.”
I don’t blame them, it doesn’t make for good messaging to admit you’re uncertain about anything. Politicians make it all the way to the top with that philosophy. However, it is important when people start making decisions based on the uncertain information that you are publishing. With carbon footprints, the people who might make decisions are investors who invest in a new low carbon start-up, or consumers who might pick one product instead of another because they think it’s better for the environment. In these cases, understanding uncertainty becomes important.
Most of my research is about new ways of quantifying uncertainty so we can improve the way we do our measurements, but the first thing I wanted to do was see what companies are doing now and whether it was actually an issue. It certainly was. Less than 1% of companies published a quantitative description of uncertainty (like “we think there is ±10% uncertainty in our measurement”), and even those that did were light on details.
Here’s what we found:
- 2,101 (79.7%) of 2,636 reports included a GHG emissions disclosure, but the vast majority were single-value estimates.
- 50 emissions disclosures (2.6% of 2,101) explicitly mentioned uncertainty related to their estimate, and 12 (0.6%) included a quantitative assessment of uncertainty.
- Firms often claim emissions reductions based on small margins that could be invalidated by unreported uncertainties.
Plenty of companies have already set climate targets, and decision-makers are now spending serious bees and ‘oney on decarbonisation initiatives. It’s a huge problem when those initiatives are being designed around non-transparent carbon footprints with large, unreported uncertainty ranges. We wouldn’t sign off on an engineering project or a health and safety decision without understanding how shaky the underlying numbers are, so why should decarbonisation be any different?
That gets me to two key final thoughts: companies and sustaniability managers who care about rigour should be frank (can I still be Garth?) about uncertainty in their data and uncertainty analysis should be a mandatory requirement of the Greenhouse Gas Protocol and ISO standards, and academics, standards bodies and consultants need to actually give people the guidance and tools to do it properly.
I’m hoping that my next piece of research will help with the second one!