Four minutes after the theft, Fr33 Aid received a donation of 0.01 bitcoins, and later that evening the first of 2 transactions with proceeds from the Texas Bitcoin Conference charity events were received, totaling 1.5 bitcoins. These bitcoins were secured yesterday, as soon as we learned Fr33 Aid’s wallet had been compromised, and the donation address on our website was updated. Please use our new address for any future donations.
Fr33 Aid’s wallet had 2 Factor Authentication enabled prior to the theft, and we are actively working with blockchain.info to investigate how it happened. They have been very responsive, and we expect to be able to update this post with more information soon.
Update #1: 10:45 am PDT, 19 March 2014
See embedded video below confirming I set up the new address using both a different device and wallet program than I had used previously.
Update #2: 12 noon PDT, 19 March 2014
Today I was able to access Fr33 Aid’s blockchain.info wallet for the first time in a few days, as their wallet functionality was out of service during that time. I had previously turned on logging, and in checking the log I discovered that a Blockchain.info Admin had approved a 2 Factor Authentication reset on Sunday, about a day after the request was made and about an hour before the settings were accessed and wallet was updated and bitcoins stolen. I was the only person with access to Fr33 Aid’s wallet, and I did not request the 2FA reset. I understand from Mandrik at Blockchain that the 2 IP addresses used for these activities (184.108.40.206 and 220.127.116.11) were both Tor exit nodes.
What We Did Right
Enabled 2 Factor Authentication for Blockchain wallet access using Google Authenticator.
Turned on logging with IP addresses for all Blockchain wallet activities. This allowed us to identify the 2FA reset that helped the thief gain access to the wallet.
Regularly checked the balance on Fr33 Aid’s address, not relying solely on email notifications. This enabled us to detect the theft and secure the recent donations before they were taken as well.
Secured the private keys for Fr33 Aid’s addresses offline and in a manner that was separate from the device used for accessing the wallet. This enabled us to access the more recent donations we received.
What We Are Doing Differently Now
We kept most of our bitcoins in our main, published Fr33 Aid address and almost all of our bitcoins in a single blockchain.info wallet that was tied to a fr33aid email address. We had thought it was good practice for donors to be able to see how much bitcoin Fr33 Aid had and make an informed decision about whether to donate or not. We recognize now that there are much more secure ways to be transparent, and we plan to continue being transparent with donors, volunteers and stakeholders in a more secure way going forward. We are in process of establishing an offline wallet system and will only use our published address as a carry-through address and forward bitcoins to additional addresses and paper wallets above a threshold. Many thanks to Jason King from Sean’s Outpost for pledging to send us a computer for offline transactions.
Fr33 Aid’s Blockchain wallet alias was Fr33 Aid. I hadn’t realized this was one of the pieces of information needed to reset 2FA. We will avoid this type of identification in future.
Our Blockchain wallet passphrase was 10 alphanumeric upper and lowercase characters but was not as strong as it could have been, with some similarities to our passwords used on other apps. We have established a new process for setting complex and unique passwords for each site and application and will change them regularly, even when 2FA is on as well.
Although the thief of Fr33 Aid’s bitcoins did not access our wallet through email or our wallet backup file, we discovered a security flaw during our investigation. Automatic email backups were enabled in our Blockchain wallet, and I was not aware that this file contained the private keys. I had been under the impression that Blockchain did not access the private keys, and I had not tried to follow instructions for restoring a wallet from the backup. In future, this and any similar feature will be disabled and I will test out the backup process and fully understand what’s in any encrypted files a wallet sends me. Edit 28 March 2014: Clarified that email/ wallet backups were not a contributing factor in this case, just something we discovered and tightened up after our investigation.
We did not check the box to prohibit access to our wallet from Tor IP addresses. In future, this or any similar box in other wallets will be checked.
We had not listed a secret phrase to help with wallet recovery, and the only contact information tied to this wallet was my well-publicized fr33aid email address and the same phone number that appears on my Fr33 Aid business cards. Because Blockchain currently requires all pieces of information provided to match in order to approve a 2FA reset request, it is important to include a unique Secret Phrase and as much identifying / contact information as possible.
Recommendations Made to Blockchain.info
1) Strengthen procedures for 2FA reset request approvals, especially in cases where there is no secret phrase on file. Require verification by all contact methods provided.
2) Log the details of 2FA reset requests and require IP addresses other than Tor exit nodes.
3) Make it possible to download a wallet backup file directly form the blockchain.info wallet app/ extension to a secure location (thumb drive, etc.) The email option for sending wallet backups is not secure.
4) When wallet users specify they want automatic email backups, a strong warning should appear that says something like, “You are about to trigger an encrypted backup file to be sent to your email. Anyone with access to this file will be able to access your private keys with only your passphrase, even if you have 2-Factor Authentication set up for your wallet. Are you sure you want to continue?”
5) Educate users about what backup files include and how the process works in greater detail, and use caution when saying Blockchain doesn’t access users’ private keys. If Blockchain was able to send the wallet backup, that means Blockchain had access to the keys at some point.
Thank you to the supporters who have made donations to our new address. As stated in the video above, I set this address up using a different device and different wallet program than we had used before.
I made this post before allowing Blockchain.info time to respond to my recommendations and before they had discussed our report with their security team; I will continue to provide updates as they respond and more information becomes available.
Update #3: 12 noon PDT, 22 March 2014
On Thursday, Blockchain.info posted an article about basic user security. It’s missing some important aspects we would recommend, including the following:
1) Keep the amount of bitcoins in any online wallet as low as the value you would keep in your physical wallet.
2) Turn on logging so that you will have a record of what IP addresses accessed your wallet.
3) Check the box to disable access to your wallet from Tor.
4) Use extreme caution if you decide to enable backups to be sent from Blockchain.info to your email, DropBox or Google Drive. Blockchain.info’s backup files contain private keys and can be accessed with only your passphrase and whatever password you have on those other services. We recommend keeping a separate copy of your private keys and disabling automatic email backups from Blockchain.info.
We have not yet heard a response to our recommendations for Blockchain.info improvements in Update #2 above.
Update #4: 23 July 2014
Blockchain.info finally responded to us. They’ve implemented new procedures for resetting 2 Factor Authentication in order to mitigate the risk of what happened to us.
Their response can be found here.
Like this post? Tip me with bitcoin!
If you enjoyed reading this post, please consider tipping me using Bitcoin. Each post gets its own unique Bitcoin address so by tipping you’re not only making my continued efforts possible but telling me what you liked.