Blog header

Database crash

Published by Oleg Gutsol · January 18th 2011

Yesterday morning we had a database crash on our hosting on Amazon. It looks like the mounted volume that contained the data got detached from the EC2 instance. This caused the database to stop responding. It would not be a big deal, but all the data on the volume was from three days ago. And the daily backups, taken from that volume also contained the three days old data.

I restored the database on the new instance. Some of the photos and profiles created during this time were lost. I was able to recover the transaction data from Paypal and apply it to the accounts, so all the awesome accounts are intact. We could go parse the server logs and rerun all requests, but I think this will require taking live servers offline for several days.

We are now replicating all data to a separate instance and backing up that data daily on a separate volume. On top of that — daily backups will be transferred to a separate server, hosted in a different location.

So your data is pretty safe now. I assure you this will not happen again.

I sincerely apologize for this problem. Please upload your photos again and register your accounts anew.
     

Login or sign up to comment on this post.

BernardoIribarnegaray
Awesome Account
Bernardo Iribarnegaray  about 4 years ago
Comment hidden
BernardoIribarnegaray
Awesome Account
Bernardo Iribarnegaray  about 4 years ago
Comment hidden
BernardoIribarnegaray
Awesome Account
Bernardo Iribarnegaray  about 4 years ago
Comment hidden

Our thoughts on photography, web development, and life. Join us for updates, interviews, reviews, and stories.

Subscribe to RSS

Popular Photos


BorrowLenses. Get 10% off your next order with the 500px gift code