I have long been a fan of Dr. Sasha Luccioni, AI and climate lead at Hugging Face. So when I learned that they would be hosting a challenge to test the energy efficiency of training models related to climate change alongside Theo Alves da Costa from Data For Good France, I immediately decided to participate.

The Team

I roped in some of my data scientist friends from grad school, who were equally happy to participate. We met during the Master’s of Information and Data Science at UC Berkeley, and despite coming from different educational and professional backgrounds, we all share a passion for sustainability. So much so, that three of our four Capstone projects focused on sustainability. Jacqueline Lam’s was an app that used computer vision to identify which garbage bin to place our waste in (recycling is hard!), another used computer vision to check air quality, and my own Capstone project, Ribbit, used machine learning to automatically identify and classify calling frogs. The final team member, Jenna Sparks, specialized in machine learning systems and environmental data analysis; she is now also a Data Scientist at Ribbit AI, the spin-off company we’re starting based on Ribbit. All in all, a very powerful team.

The Task

The goal of the Frugal AI challenge was to obtain the highest possible accuracy, with the lowest energy consumption.

There were three possible tasks (source: https://frugalaichallenge.org/about/)

1. 📝 Detecting climate disinformation 📝: based on text from news articles.
2. 🔥 Classifying regions at risk of wildfires 🔥: based on image data gathered via satellite.
3. 🌳🪓 Detecting illegal deforestation 🌳🪓 : from bio-acoustic data recorded in the jungle.

Funnily enough, despite three of us having experience in computer vision, we selected the text challenge. We chose this task because we were all familiar with Natural Language Processing, and felt we could contribute equally. We were also curious to know the type of disinformation that would show up in the dataset.

The Thought Process

Where to begin? We had always focused on traditional evaluation metrics: Accuracy, F1, precision, and recall. But now, this wasn’t enough; we couldn’t leave our computers running for days on end, and fine-tuning until we got to the perfect score. Also, since the challenge focused on frugality, throwing the largest LLM model at the problem wasn’t ideal, since the energy consumption would be very high. So we decided we would take the following approach:

1. Analyze the leaderboard to understand the competing models’ performance.
2. Establish a simple baseline.
3. Fine-tune low-energy models.

1. Analyze the leaderboard to understand the competing models’ performance.

The challenge offered the option to submit our models to the leaderboard, so before selecting our models, we looked at the competition. Out of the 42 submitted models, the mean energy consumed was 3.34 W, and the mean accuracy was 0.60. We now knew where to start, with these target metrics in mind.

2. Establishing a simple baseline 💡

We wanted to know the minimum energy expenditure of a baseline model, so we evaluated a model that would randomly predict the class. To do this, we used the original dataset, without balancing the data, since our focus wasn’t on accuracy but energy expenditure. This yielded a very low accuracy of 0.14 (to be expected), and it consumed very little energy: 6.36e-05 Wh. That is approximately equivalent to keeping a 60W incandescent LED light bulb on for around 0.038 seconds. That was pretty impressive since model inference took 15 minutes on a computer with the following specifications:

Platform system: Linux-6.1.85+-x86_64-with-glibc2.35

Available RAM: 12.675 GB

CPU count: 2

CPU model: Intel(R) Xeon(R) CPU @ 2.20GHz

3. Fine-tune low-energy models

With over 900k models to choose from on Hugging Face, the task seemed daunting. But luckily, these models can be filtered by task, and other variables. We filtered by models that were low on carbon emissions since low emissions models and low energy models tend to be correlated.

We noticed that many BERT models appeared in this category, so we took a similar road. We chose RoBERTa because of our previous experience with it, ModernBERT ( December 2024 paper) because of its faster processing compared to BERT, and DistilBERT because of its smaller size compared to BERT. We also chose ELECTRA, which replaces tokens with alternatives generated by a small generator network instead of a masked language model.

However, acknowledging that all of these models were LLMs, we also selected smaller models that were likely to have an ever smaller energy consumption. To do that, we chose unsupervised learning techniques, such as centroid similarity and K-nearest neighbors.

The Data Preprocessing

Remember when I said that we had target metrics in mind? Those targets changed when we watched the Frugal AI webinar: the organizers were looking for accuracies of 0.9 and above for the text challenge. Obtaining an accuracy that high with an unbalanced dataset might be feasible, but we knew it wouldn't generalize well. So the first step, before even training our models, would be to balance our dataset.

Balancing our dataset

To do this, we performed EDA on our data, noticing a class imbalance, with the “Not relevant” category (No relevant claim detected or claims that don’t fit other categories) being the largest class, and the “Fossil fuels needed” category ( Claims promoting fossil fuel necessity — We need fossil fuels for economic growth, prosperity, and to maintain our standard of living) being the smallest, with only 4% of the data having those labels. As someone interested in climate change, it pleased us that this was the smallest class, but as data scientists, that made our work more challenging.

We decided to augment the dataset by performing back-translation from English (the language the labels were in) to Spanish or Chinese (languages spoken by the team) and back. We settled upon using the English-Spanish-English translations generated with the MarianMT model. We did not want the synthetic data to overpower the non-synthetic data, so the target number of sentences augmented per category was 2x the smallest category.

The Results

Our best-performing model was the fine-tuned ELECTRA:

Accuracy: 0.7333333333333333
Energy consumed (Wh): 0.9430491292312838

We weren’t able to reach the target Accuracy of 0.9. However, the energy consumption for our model was lower than the mean energy we had seen in the leaderboard (3.348833 W), so we were happy enough to submit our model. It was also performing well, with few type I and II errors:

How Did We Compare to the Other Participants?

I know you were expecting me to say that we won the challenge, but we did not. The winners were EURECOM, with an accuracy of around 0.6 and an energy consumption < 0.1, based on the results shared by the FrugalAI team at the Forum for Sustainable AI. They did however, also use a BERT flavor: Sentence BERT (t5-large).

How to Be More Eco-friendly When Training AI Models

To sum up, while we did not win, we enjoyed participating in this competition. As data scientists, we’re usually focused on evaluation metrics such as accuracy, precision, recall, and F1. However, training a model doesn’t happen overnight; it usually takes several nights of hyper-parameter tuning, leading to high energy consumption. If we want to tackle climate change with AI, or at least reduce our carbon footprint, we need to take steps towards being more frugal and mindful in our model training and fine-tuning.

If you want to incorporate this practice into your code, there are 4things you can do:

1. Use smaller models

We all love LLMs, but training and inference costs can generate a large carbon footprint. The release of DeepSeek showed promise, due to its much faster training time, but the Chain of Thought used at inference time is computationally more expensive. So while the jury is out, focus on smolLMs.

2. Use CodeCarbon to measure the energy consumption and the carbon emissions of the model

We measured our energy efficiency by using CodeCarbon, which is pretty straightforward, as you can see below in this example adapted from their documentation:

from codecarbon import EmissionsTracker

# initialize tracker
tracker = EmissionsTracker()

# start tracking
tracker.start()

# fit your model (or any other computing-intensive process)
model.fit(x_train, y_train, epochs=10)

# stop tracking
emissions: float = tracker.stop()
print(emissions)

3. Train your model in regions with renewable energy

• Google Cloud is powered by 100% renewable energy (for most of the day) but seeks to expand this to a 24/7 service by 2030.
• AWS is powered by more than 95% renewable energy in some regions.
• Azure’s target is to run on 100% renewable energy by 2025.

4. Create a Model Card for your HuggingFace model.

Make sure to include the following:

• Environmental Impact
• Limitations (including biases and ethical considerations)

If you’re unsure where to start, use our Model Card for the challenge as a guide, or explore our GitHub repo.

Recently, fellow MIDS student Sue McTaggart and I had the excellent opportunity to discuss diversity issues with three great female leaders in STEM. As DEIBJ (Diversity, Equity, Inclusion, Belonging, and Justice) representatives for the I School, we wanted to provide women interested in technology with access to people at the top of the ladder who have been advocates for diversity and women in tech. So Sue contacted Angela C. Williams (SVP, Chief Information Security Officer with UL Solutions), Jane Harper (Associate Vice President Info Security, Risk Management & Business Engagement, Eli Lilly and Company), and Sam King (CEO of Veracode), and here are some of the lessons we learned from them.

1. You will face several challenges being a woman in STEM, but your network will help you to overcome them.

“Imposter syndrome only seems to show up when you’re doing good things. When you’re not accomplishing anything […] [it] is nowhere around, but when you’re doing great things and you’re on the precipice of even greater things, it shows up. So sometimes, you also need to have a brag book, […] a scrapbook of all of the accomplishments that you’ve had over the course of your career, all the kind notes and the kind words and the acknowledgments, you just copy-paste them into that book and keep it so that when imposter syndrome shows up, you flip through the book and you remind yourself ‘oh I have done some great things.’” — Jane Harper

Imposter syndrome is a common challenge faced by women in STEM fields. We can find it hard to believe in ourselves, especially when we belong to a minority and have to battle stereotypes. This intersectionality may lead to women who are also POC not being hired or heard. One way to overcome this is by surrounding ourselves with people who believe in us and who want us to succeed. Our network is key in both building our confidence and pushing us forward. When this isn’t enough, we can boost our confidence with affirmations, like “I can do this, I am worthy, and I have accomplished many things”. We can put ourselves in the position of someone that is looking to hire, and read our resume with that lens. All these strategies can allow us to recognize when imposter syndrome shows up, address it, and then wrap it up.

2. Both top-down and bottom-up approaches are needed to increase the number of women in STEM.

“[We need to] cast a wider net than just STEM when looking to encourage women to get into the field.” — Sam King

Women are significantly underrepresented in STEM fields; in fields like data science, fewer than 30% identify as women. Getting to 30% is ideal because, at that percentage, people start to feel a sense of belonging, as you can test for yourself in this simulation based on Nobel Prize-winning game theorist, Thomas Schelling’s work. Top-down approaches to increase the number of women in STEM are actions like recruiting more women in C-suite, or senior, positions who, in turn, will hire more women. Bottom-up approaches include getting more women interested in STEM from an early age so that they will choose to go into STEM-related fields. Both types of strategies are necessary, if we want to increase the number of women in STEM.

However, women in STEM don’t necessarily need to have pursued a STEM degree because all kinds of skill sets are needed to be successful in this field, beyond the technical skill sets. For example, communication, marketing, and teaching are some of the skills that Jane Harpers’ employees at the Information Security, Risk and Business Engagement for Eli Lilly and Company hold. CEOs at cybersecurity firms even hold a degree in English. So if you’re reading this, and are interested in STEM but afraid that you don’t have the skill set to pursue a career in this field, maybe this is the sign you were looking for.

3. We need diverse strategies to elevate women inside tech companies.

“I used to say I’m not looking to be a trailblazer. I want to plow a runway. I want a runway wide enough for a 747 to land. So, women are coming in the door single file, we should be coming in masses.” — Angela C. Williams

Some of these actions are

a. Calling in women with a specific profile and asking them to consider applying for the job, even when these women try to talk themselves out of it. We don’t need to check every box in order to apply for a job.

b. Acknowledge the profound work women are doing for example: “so when Stacey has said something that is profound and then Rick repeats what she says and he is now the best thing since the microwave, with no acknowledgment for Stacey. I learned to say, Rick, it’s awesome that you agree with what Stacy said, or, it’s awesome that you built upon what Stacey said.” Jane Harper

c. Tracking how many women are at the company allows people in senior roles to evidence the gender imbalance and address it by hiring more women

d. Creating a Diversity and Belonging Group, which creates awareness.

e. Creating diversity-friendly programs, benefits, and policies.

f. Partnering with groups that promote diversity, like PowerToFly.

g. Creating a remote-first culture that is more family-friendly.

4. Storytelling is key for a leadership position

“When you’re in a leadership position where it isn’t just what you do, it’s what your team does or what your company does right, those outcomes are influenced not just by the actions you take in your sphere of control, but by how you can influence other people to do as well.” Sam King

Great communication skills are key to being a great leader and mastering the art of storytelling: if you can wrap that into a story that people can connect to emotionally, you’re going to have a much easier time doing it. Understanding your audience will help you pivot in order to get what you need. Using analogies allows leaders to be more relatable so that people will understand why a certain decision will impact them. Reading the room is key to understanding the whole message.

Along with excellent communication skills, a leader needs emotional intelligence. You will have conflict over the course of your career; emotional intelligence helps you to navigate around it. Listen with an empathetic ear, and lean all the way into actually understanding your team. Finally, a leadership position is also about creating a network and influencing the influencers.

5. To get to a C-role, you must advocate for yourself.

“When you’re that rockstar, sometimes a company does not want to unplug you from being the rockstar because then, who’s gonna be the rockstar when you go off and do something different?” Jane Harper

Getting to a C-role takes time and patience, but above all, a network of cheerleaders, mentors, and sponsors who can advocate for you. However, sometimes you need to advocate for yourself. You have to know your worth, especially when negotiating a position of higher responsibility inside your company, because your immediate boss may not want to lose their star player. You can talk to different leaders in order to get feedback, in order to validate your work and don’t be afraid of shopping around for other opportunities in other companies.

Networking, storytelling, and advocating for ourselves are just some of the ways for women to get ahead in the tech industry. But we also need allies, cheerleaders, and mentors. Having more women at the top implies helping each other out, recruiting people from diverse backgrounds, and getting girls interested in STEM from primary school onwards. We still have a long way to go, but having leaders such as Jane Harper, Sam King, and Angela C. Williams will make the path much easier.

This blog post was adapted from a panel, find the full panel discussion.

Juliana Gómez Consuegra is pursing an a Master of Information and Data Science at UC Berkeley’s School of Information. She also serves as an I School DEIBJ working group member, where she participates in committee meetings with faculty and staff, engages with prospective students in webinars and in-person events like prospective student visit days, and develops and manages student-led DEIBJ initiatives.

5 Things I Learned from Leaders in STEM was originally published in BerkeleyISchool on Medium, where people are continuing the conversation by highlighting and responding to this story.