How to Remove a Secret Token from Your Git History
This article was written by Yonatan Kra, Vonage Software Architect at Vonage. Originally published on the Vonage Developer Blog.
Introduction
In this article, you’ll learn how to safely remove exposed authentication tokens from your Git repository’s entire history using BFG, a powerful cleanup tool. You’ll also understand the security risks of committed secrets and master the step-by-step process to sanitize your code repository and protect your organization’s digital assets. Let’s get started!
The Unexpected Security Alert
Picture this: You’re a developer, deep in the middle of a meaningful sprint, when suddenly a JIRA ticket lands in your inbox. The subject line sends a chill down your spine: “Critical Security Issue — Exposed NPM Token in Repository History.”
The automated security scan has uncovered something alarming — an authentication token accidentally committed to the repository years ago, buried in commits from 4, 7, and 8 years back. It might seem like ancient history, but in the world of cybersecurity, old secrets can be ticking time bombs. And your CISO can be very insistent on solving such issues (with merit…).
Ok, you can stop imagining because this happened to us at Vonage. Thankfully, you can learn from our experience.
The Hidden Risks of Exposed Tokens
An exposed authentication token is more than just a minor oversight. It represents a significant security vulnerability that could potentially:
- Allow unauthorized access to private package registries
- Expose sensitive company infrastructure
- Provide a potential entry point for malicious actors
- Risk company intellectual property
- Potentially violate compliance requirements
The reputational damage of a security breach can far outweigh the momentary convenience of a hastily committed token.
The first thing you’d like to do in such a case is to “rotate” the secret key(s) involved. Just null them and create new ones. But the risk doesn’t end there, because as someone put it in a JavaScript group discussion — old obsolete keys might hold a clue to the key generator’s algorithm itself. I’m not a cyber security expert (not even close), so I’ll take the experts’ words on it. How do we get rid of the exposed key in the repo then?
Enter BFG: Your Git History Cleanup BFF
The solution? BFG (Big Fast Git), is a powerful tool designed to clean up Git repository histories quickly and efficiently. Here’s a step-by-step guide to token removal:
- Tell everyone who’s working on the repository to hold their work for a few minutes until you push your changes.
- Download BFG and cd into its folder.
- Clone the repository with a mirror:
git clone --mirror https://github.com/your-name/your-repo.git- I suggest doing it twice to two different folders so you will have a backup of your “old” repository if something goes wrong.
4. Create a tokens.txt file with the replacement syntax:
old_token1==>PLACE_HOLDER1
old_token2==>PLACE_HOLDER2
...This will tell BFG to replace any instance of the old tokens with their placeholders.
An example of this would be:
ab23234-89723c-aa7b7c==>${NPM_TOKEN}ab23234-89723c-aa7b7c==>${NPM_TOKEN}
5. Run BFG with the replacement file (make sure you are in the repository’s parent folder):
java -jar bfg-1.14.0.jar --replace-text tokens.txt your-repo.git --no-blob-protection6. Clean up the repository:
cd your-repo.git git reflog expire --expire=now --all && git gc --prune=now --aggressive7. Check the token was replaced by cloning the local mirror and “grepping” it:
cd .. && git clone your-repo.git && cd your-repo && git grep -n "REPLACE _WITH_YOUR_TOKENIf this returns 0 results, it means it worked.
8. Force push the changes:
cd your-repo.git
git push --mirror9. Tell your teammates to fetch and apply the changes:
git fetch git reset --hard origin/mainThat’s it. Your repository is now clean. From now on, I guess the same tool that alerted you about the tokens can raise the alarm during a PR process before the token “spreads” to the main branch. Make sure it does 😉
Pro Tips for Token Management
- Never commit tokens directly to repositories
- Use environment variables or secure secret management tools
- Implement pre-commit hooks to prevent accidental token commits
- Regularly audit your repository history
- Rotate tokens periodically, especially if you suspect exposure
Final Thoughts
In an era of increasing cybersecurity threats, proactive repository management isn’t just good practice — it’s a necessity. Tools like BFG provide developers with powerful mechanisms to clean up historical mistakes and maintain the integrity of their code repositories.
While we’ve focused on token replacement, BFG is a versatile tool capable of:
- Removing large files from repository history
- Cleaning sensitive information
- Reducing repository size
- Sanitizing commits before open-sourcing projects
If you adhere to the reasons you should use conventional commits, BFG can allow you to spice up your old commits with some meaningful version and changelog text.
Remember that in software development, what’s committed isn’t always forever, thanks to tools like BFG.
Let us know what other Git issues you’re having on our Vonage Community Slack or send us a message on X.
For the most up-to-date version of this article, please refer to the Vonage Developer Blog
