Avoid common causes of data loss and system crashes

Published by Chris August 1st, 2006

We see a spike in the number of system failures each year during the summer months. This summer has been no exception with the very high recent temperatures.

hard driveWe value our clients and want to work with you to help reduce the possibility of disruptive system crashes and potential loss of data for your office.

Guidelines to reduce data loss or disruption for your systems

  • Make sure your backups are complete: Make sure you understand where your firm’s critical data is stored and that all your firm’s critical data is backed up on a daily basis. Every drive letter that you store data on, databases, and your company’s email data are a good starting point on what your daily backup jobs should include.Also remember that it is important to backup your user created data and your server “system state” and programs per server. It can take alot of extra time to reconstruct all the programs and configuration settings on a failed server - even if you have a backup of your user created data. In our experience, most small firms do not take the time required to backup anything other than their data - leaving their operations open to a lengthy disruption in the event of server crash.

    The configuration of your backup jobs should be re-checked when the composition of your systems change, especially when new hardware, users or additional software is installed.

  • BaysideNetworks.com, Inc. does not routinely check your backups; we check client backups only when our clients specifically request it. It is your responsibility to monitor the effectiveness of your backups on a daily basis and report any related issues to us that you might need assistance with.
  • Have your data backup system reviewed and tested by an expert at least once per year, or whenever your system configuration changes. Testing might include steps such as:
    • Restoration of a few files from your backup, especially database (SQL) and selected emails.
    • Review and comment on: your firm’s daily backup log, the number of backup targets and the completeness of your backup routine – i.e. does it include all data that is obviously important for your office’s operations.
  • Be sure to keep at least two off site backups of your data – ideally one off site backup should be a week or less old and you should maintain another separate quarterly off site backup as well. You should log the rotation of your backups logs in writing and be sure to store your off site backups in a safe location especially if they contain confidential information.
  • Safety in numbers and backup depth: You should have at least 10 copies of your firm’s critical data at any given time. This might seem like a lot of extra copies of your data, but if something goes wrong with your backup process and you don’t recognize it quickly you can quickly find that all of your backups have been overwritten, leaving you with no backup. This kind of scenario isn’t exactly common but it does happen. Also, sometimes you won’t recognize that a file has been damaged or deleted right away so restoring from an older backup might be your only option to restore that information. As a general guideline for a small business, you should make sure your office has one backup for each day of the week, then special separate backups that are rotated separately for the end of the preceding three weeks and the preceding three months. You should take the time to record a memo on how backups are rotated and stored within your company and who is responsible for this important aspect of your company’s operations.
  • Your firm should record (log) the amount of data that is backed up each day, and request support from your support provider (usually BaysideNetworks.com, Inc.) for each day that the backup fails. The amount of data for each data source (email, accounting data, databases, etc) within your office should be recorded and any unexpected deviations should be researched and addressed. Ideally your backups should be logged by someone other than the person that swaps your backup devices each day. The best approach is to post a written log, listing the date of each backup and any problems that were noted in an obvious location within your office. A good logging procedure is the single most important aspect of protecting your data.Sure, we can check your backup on a specific day but that very likely won’t be “the day” that your system actually fails. A good logging procedure can help ensure that future problems that might develop with your backup systems are quickly identified and resolved. It’s situations where a backup procedure has failed for many days in a row and “no one knew about it” that create the most risk—a good logging procedure can substantially reduce this type of risk.
  • Make sure your servers incorporate fault tolerance features, such as “RAID” and that the health of your server systems are checked regularly. Are the fans spinning in your servers? Have any of the RAID drives failed?  Sometimes partial failures can leave a system more vulnerable to data loss – make sure your systems are checked at least a few times a year for obvious problems.  Dealing with a simple and comparatively simple problem now can sometimes save extensive disruption and potential data loss later on.  Restoring from backup is a last resort and usually even if your data is backed up it will still be disruptive and time consuming to restore a failed system.  Keeping your systems in good health to begin with and replacing them within a reasonable obsolescence cycle are helpful toward reducing data loss.
  • If your office uses a single file server – take the time to understand the special risk that situation creates. With one server, the loss or failure of that system can create a very disruptive situation where your firm’s “security domain” is lost.  Indeed you may have a data backup, but if the domain is lost it can be time consuming to recreate and can substantially amount to a time consuming rebuild of your system setup.
  • Temperature monitoring: install a temperature alert device for your server’s location. The alert device should be set to call you if the temperature in your server room rises above 85 F.There are many good temperature monitoring devices available, search on “FA-I-CCA Intermediate FreezeAlarm” at Amazon.com for a typical device. Basically if it’s uncomfortably hot for you, then its probably too hot for your server system(s).   Most server failures definitely seem to happen on weekends too – when no one is around and A/C systems are normally turned off.

BaysideNetworks.com, Inc. is ready to help your firm with all of the above areas. Please give us a call anytime to discuss your service needs. Working together we can help protect your company from frustrating data loss and potential system downtime.

Thanks,