TotalBalance
I recently had a system write error on the USB drive that holds my PB brains. Unfortunately, it happened while PB was accessing the drive and a PB database corruption resulted that required PB Support to repair the database and email it back to me. Harlan & his support team did a great job diagnosing & fixing the issue and all is good. Thanks again Harlan. If interested, here's the post.

The issue and associated downtime for me did get me to think about how I'm backing/restoring up my brains.

With the advent of USB drives attached to multiple USB 8 port hubs plugged into ones laptop there does seem to be a re-birth of system write errors that used to be a thing of the past associated with internal hard drives. When they did happen, in the "old" days, it was symptomatic of needing to replace the hard drive. With USB drives, it seems more to do with the attached USB hubs and the OS temporarily "losing" track of what's connected. Drive connections get lost for a moment only to auto reconnect a few moments later. If drive activity just happens to be going during those moments, the sys. write errors occur.

Regardless, especially for those like me who've decided to go with the "living in one big brain" with attachments stored "internally" approach, a large USB drive seem to be a necessity for me. The benefits of portability, storage size and enhanced PB search functionality with this approach are powerful incentives. I suspect many PB users, even those who prefer multiple smaller brains & attachment links, use USB drives to store they're brains. Yet, a failure like I've just had and the associated downtime pretty much negate a lot of it.

Moving forward, is there anything PB can do in future versions to allow the user to recover this types of _db corruption without having to send it to support?

If not, or at least in the meantime, what type of automated backup routine/recovery plan should I:
  • Create periodic brainzips of my master brain?
  • Use a 3rd party backup program to make incremental or time specific backups of the entire folder structure.
  • Something else?
After doing a quick search/review of this topic, this post suggests both a 3rd party backup program and periodic brainzips. Sounds pretty bulletproof. Since it's posting date (12/07) and now, are there better strategies or is this still on target?

What do others think? Harlan?

Thanks in advance.

Quote
Antonio68
I have one big brain with all attachments stored internally. Actually my brain is more than 2GB and more than 2,000 thoughts.

The brain is located in my desktop at work and my labtop at home. The two brains are synchronized with Live Sync. It works perfectly unless you do not open the brain simultaneously in the two locations. In addition is important to wait complete synchronization before opening the brain in the second location. Not a problem for me since it takes me 30 minutes to go to work!

In the two location I also do sequential full backup of the brain file and the brain folder ( I use Cobian backup).

The system works fine. I do not need any USB stick.


Quote
JonnyB
I use Live Mesh that both automatically syncs changes with an online desktop but will also sync with another computer or 2 as well.  This keeps an ongoing back up online.  I also backup everything using synctoy to an external hard drive so I am covered on many fronts.
Quote
Antonio68
Hallo Jonny, I tried Live Mesh too. At the beginning I was thinking it would be better than Live Mesh, since it saves the brain on the cloud and I don't need to keep the both computers on when synchonizing.
It was working fine with a little brain but was not able to synchronize my main brain.
How big is your brain that works with the Mesh?
Quote
Harlan
Be careful with folder sync software going in two directions. It WILL corrupt your Brain and you WILL lose data if you open the brain from both locations at the same time or do not allow the sync to complete.

I recommend BrainZips or a one-way sync to an external hard drive.
Regards,
-Harlan
Quote
JonnyB
1.75 GB size

Live mesh can handle up to 5 Gb presently (rumors that that will go up like Skydrive to 25 GB). 

I don't back up Brainzips online as that would require a vast amount of time and tie up one's internet everytime one does a backup.  However, I do backup Brainzips (which I zip weekly) along with my complete Brain onto an external hard drive. 


Click image for larger version - Name: Capture.JPG, Views: 535, Size: 23.88 KB
Quote
Darkstar

You zip brainzips? You do know that brainzips are ALREADY zip files--- so you are doing anything but growing them (unless you are bundling a lot of brainzip files together--- which grows their size, but creates one zip holding the bundle of zips). Now, there are some archival/compression formats that can compress zips files, so you if you are using those, then, more power to you.

-Darkstar
Quote
TotalBalance
Thanks all for your responses. Great advice and I'll definitely be modifying my daily backup routine.

I guess my root question really boils down to how best to recover database/content integrity after a system write error occurs under the above scenario. For example, MS Office Word & Excel both have an auto "recovery" option pop-up to deal with these type of sys. hw failures. Sure you lose some of your work, on the specific document you where working on, but that's all - the doc. is recovered from the last know good version. Is there anyway to have a similar PB recovery option (either within PB or using 3rd party products) so all you lose is what you where working on before the write errors and not the entire brain. By that I mean not even being able to access the brain without PB Support intervention & manual db fix? Let me know if I need to explain further.

P.S. my master brain is over 80G with 8K in thoughts, 8.9K links, 2.8K in notes, 2.3K internal and 200 external attachments, & 2.5K URL Attachments - if that matters.

Thx. in advance.

Quote
Harlan
FYI, we very rarely need to be involved in recovery of data. PB makes an ongoing backup of your data as it works, so there are always two copies in your _brain folder unless one gets corrupted. In the vast majority of cases, when corruption occurs it is detected by PB and an auto-recovery is initiated.

BrainZips placed on external drive are to safeguard against total loss of your drive (IE complete disc failure or total loss if your laptop gets stolen, etcetera).

Word, Excel, and other document based file formats are of course much simpler since the whole thing is loaded into memory at startup and then saved out at shutdown (or at interim points), renaming the prior version - effectively creating a backup. Obviously, PB can't do that since your Brain wouldn't fit into memory all at once and since the time it would take to load and save your entire Brain all at once would be prohibitive. Your Brain is more like a set of interconnected files (a database in fact) than a single file.
Regards,
-Harlan
Quote
TotalBalance
Thanks for the explanation Harlan. I've no doubt PB support "rarely need to be involved in recovery of data". Regrettably, my comfort level is still on shaky ground. I've never meant to suggest it's a PB issue, unfortunately sys write errors do have the potential, as previously seen, to cause PB db issues.

The reason for this post was in hopes there was a backup procedure for my brain I could implement to reduce those rare occurrences.

It sounds like the best I can do (from a software perspective) is to setup automated periodic one way PB file synchs and brainzips. If I experience another sys. write error, while working in PB, resulting in _db corruption, I'll revert to the last known good backup which, unfortunately, could mean up to a day's worth of work lost. Something I hope never happens again but the possibility does exist.

I'm also looking into how to reduce these sys. write errors. Perhaps buy the largest USB drives available (1T+) to reduce  the number of drives attached to my USB hubs while allowing a larger number of PB backups/brainzips to be saved. Or, perhaps a cheap pc, attached to my home office network, whose only purpose is to hold all my USB drives, except the one used for my brain. That I'll attach directly to a laptop usb rather than a usb hub.
All food for thought
Presently I'm using two USB 8 ports hubs with 14 devices (7 of which USB drives) plugged in. I guess my old Dell XP box just "forgets" what's connected from time to time

System:
WinXP (SP3)
Media Center Edition - Version 2002

Computer:
Intel Pentium M (Centrino Mobile Technology)
Processor: 2.0 GHz
1.99 GHz, 2.00 GB of RAM
Physical Address Extension

Quote
grsilverman
I'm also worried about the integrity of the data, as I'm in the process of moving my entire life into PB.

May I respectfully suggest that you look into a Mac, since they seem to have less issues like these. I've never had an issue with read/write corruption with multiple USB and firewire drives. Also, you might want to look into a mirrored drive configuration. Time Machine backs up hourly, and I use Mozy to back up over the Internet.

You can also use mobile.me or dropbox to keep brainzip files. If you synchronize your PB directory, you will have automatic, off the premises backup on a continual basis, that you don't have to remember. I'm going to be very very nervous on my first few trips out of the office with my laptop, when I'm disconnected from these safety nets.
Best,

George Silverman
Author, "The Secrets of Word-of-Mouth Marketing"
http://www.mnav.com
Quote

Newsletter Signup  Newsletter        Visit TheBrain Blog   Blog       Follow us on Twitter   Twitter       Like Us on Facebook   Facebook         Watch Us on Youtube  YouTube       

TheBrain Mind Map & Mindmapping Software     Download TheBrain Mind Mapping Software