Continuing an occasional feature
This just came over the transom. Vendors, take your best shot at configuring a good solution.
Non-vendors are welcome too. I’ll email him a customized StorageMojo take after the bulk of the comments come in.
Welcome to the real world, Neo.
Vendors often wax eloquent about the SMB market when enterprise IT dollars dry up. But this note is a good example of the problems vendors face – mostly resellers – in servicing this level of customer.
- Budget. $20k won’t pay for a direct sales visit.
- Mission. Backup, archive or both? Would you recommend a RAID system for either?
- Education. How likely is it that the Major’s requirements can be met with a disk-based solution?
- Integration. The writer is a busy guy who does sys admin work as a sideline to his real work. My sense is that most non-tech organizations with fewer than 300 employees don’t have full time sys admins. So it either needs to virtually install itself or come with someone to install it.
- Service. This is a self-serve client. Hardware repair and upgrades must be simple. Software updates ditto.
Also, he doesn’t mention data retention and chain-of-evidence issues, but sometimes cases come back year later for judicial review or on appeal. Keep that in mind as well.
Here’s email in (mostly) his own words
I edited a bit for clarity.
We are a Sheriff’s Office with two-hundred sworn members. I run the CSI unit full time and supervise the Evidence Custoodian and Forensic Services Tech. I also process CCTV video and digital imaging for the county.
I have a digital photo room and a video processing room. The digital photo room has a stand-alone agency PC that is not plugged in to the internet/intranet. It is plugged in to the power outlet, that’s it. I have a 130 GB external hard drive for that system. I download digital images to the software on that PC and download the monthly folders to gold plated DVD’s. One DVD stays here at HQ, the second goes to the office at the courthouse.
The video processing room has a stand-alone Avid Adrenaline PC from Ocean Systems [ed. note: a leading vendor of forensic video analysis systems]. [The Avid] is also not on the intranet and the memory is all internal on it’s own drives.
My Major met with me this morning asking me about getting a RAID backup system. He has about 20k from a grant and wants us to get the best bang for the buck. He also does not want to get nickeled and dimes to death several years down the road when we ask him for more money to upgrade the RAID. He is worried about support for the RAID beyond the typical five year warranty/support period.
What advice can you provide me about using a back-up system for both PC’s? I told him that we are currently playing Russian roulette with the systems and that I do not want to be the person that has to tell the Sheriff that the system has failed and we do not have it backed up.
I can ask for more information if deemed essential, but I think there’s enough there to make informed estimates. This is why good resellers are worth their margin points, when our corporate overlords permit them.
The StorageMojo take
The only difference between this scenario and the issues I see here in Smallville is that the budget is larger and the writer is more sophisticated about storage than most small business owners. But as the generation and use of digital data becomes pervasive we have to have solutions that reach this level of problem for even lower cost.
After all, is this so different from where the average family will be in 5 years?
Courteous comments welcome, of course.
RAID is not an archival strategy. It protects against a disk failure. Not operator error. Not application corruption.
US$20K will acquire a reasonable controller-based or array-based four- to six-terabyte DAS storage array from various second-tier vendors (and potentially more), and an LTO-4-class tape drive and a reasonable-sized stack of blank tapes for data archiving.
If you need shared storage (NAS or SAN) for multiple systems, costs go up and/or the I/O bandwidth goes down.
LTO-4 tape gets you encryption and WORM capabilities and a relatively long media lifetime, all of which can be of interest in this market segment. LTO is sometimes considered high-end for SMB, but it is also a very reliable archival technology.
Issues to be resolved here include the rate of data change and data increase, and the compatibility of the host platform and software with the RAID array and with LTO Ultrium or analogous widgets, and any host archival software that may be required.
Get the systems that need backups on a wired or wireless private backup LAN for backups only without internet connectivity. Get several laptops with full disk encryption and use them as backup targets. 500GB full disk encryption drives are available. The data set didn’t look that large from the above description. Rotate the half the laptops daily or weekly to an offsite location. So several laptops have the same data onsite and offsite. Get a good replacement plan from the laptop vendor. If MS windows is the source OS then use the sync application from Microsoft to make copies of the data on the target laptops. Use rsync for Unix/Linux. Use the built-in schedulers for automation. Some scripts are required and some purging rules for old data is also a good idea. So figure perhaps 6 laptops with nextday onsite hardware replacement at $2000 each and $8000 for the setup and scripts and any other work that needs doing. More copies of the data is probably better than an elaborate RAID setup in this case. The crypto should also avoid alot of headaches.
Windows Home Server is a good answer.
The backup solution is simple, brilliant, more efficient than nearly anything else, and will swallow any typoe of storage you throw at it. The disks are world readable in the event of backup system failure. A private LAN, or Home Server for each machine.
It’s the first new idea in backups in years, and it’s US$99
http://neuralfibre.com/paul/it/windows-home-server-pp1-im-impressed
has all teh juicy bits.
Interesting comment about this being the typical family position in 5 years. I don’t think it will be that far away.
Given the indicated small volume of data I would suggest using some commodity NAS devices to provide the backup and archiving. Intel among others retail small NAS devices with 4TB + of storage for under $1K.
Two or three units provide online backup capacity and archival storage. The units can be reasonably maintained (drive replacement) by the admin and support at minimum RAID1 and RAID5.
More focus is needed on the backup and archiving software and the processes that are used to manage those operations. In this sort of situation I can see an argument for spending some of the budget to get a specialist in backup and archiving to look at the system and the data itself and tailoring an automated solution. Nothing like having the systems back themselves up with time based recovery points rather than having to rely on the poor admin to kick things off.
The need is for:
Protection against drive failure
Protection against site failure (specified for the images, assume for video also)
Doesn’t touch a network
The solution that came to mind for me is small, cheap, self-contained RAID boxes. For the image machine: Have two at HQ (raid 5, identical data on each box), two at the courthouse (identical data on each), with 2 portable HDs to move data from site to site (in case one fails, another is nearby). May need to buy a couple firewire PCI cards. For the video machine: If backing this data up isn’t so critical (protecting against site failure), then four raid boxes, otherwise two here and two there, what have you. Lets assume its not critical, so four raid boxes just for storage
In 15 minutes I found the LaCie 4big Quadra boxes, 6TB for $1.5k, with a 3 year warranty and extra replacement drives for $250 each.
Solution Explained:
The image machine would have 2 of these boxes as raid 5, each containing the full set of data, effectively 4.5 TB of raid 5+1. A configuration like this would also be at the courthouse. Data backed up between sites using the a portable HDD as often as desired (with a spare around just in case). Probably need a firewire card for each machine.
The video machine would have 4 of these boxes as raid 5, so 18 TB of raid 5. Or 12 TB raid 0+1, so any drive or box can fail with no damages. Also get an extra firewire card to make sure to have enough ports.
The drives have 3 year warranties (‘eh, it’ll do), but buy 8 extra drives ‘so i can swap them the instant one fails’.
Everything is plug and play, from configuring cards, to storage, to backups, moving files is simple. A highschool kid who built his own desktop once could do it start to finish in a day.
Total purchasing needs:
2x 500 Gig external USB SATA drives ($125 ea)
8x 4big Quadra 6TB boxes ($1500 ea)
3x firewire PCI cards ($50 ea)
8x replacement drives for the Quadras ($260 ea)
Total cost:
$14,480 (or, if you add 10% for pessimism tax, $16k)
Thats 4.5TB of ridiculously well protected data and 18TB of reasonably protected (or 12TB of well protected) for under $20k, as configurable and manageable by a competant, but entirely untrained, individual.
I think that LabRat is going in the right direction with the idea of easily portable data bricks with a few caveats.
If we can run a private network between the two machines, a NAS used as a backup datastore would be the ideal solution.
To protect against the basic stupid user error deleting stuff by accident, I would use either OpenSolaris or Nexenta as a front end for managing the content of the data bricks with the automatic snapshots activated and set to auto expire at reaching 80% capacity. You get pseudo backups on top of the RAID level protection.
The advantage of the Nexenta approach is that it comes with a easy to learn web management interface whereas the OpenSolaris approach will require that you do a little bit more manually. (cost? consulting?)
Using ZFS also removes any hardware dependencies. You can pop SATA disks into any computer with a ZFS stack and mount the zpools, and consequently the ZFS filesystems. From a lifecycle perspective I think that this is a more judicious approach that depending on being able to replace a failed RAID card or enclosure x years down the line.
For data bricks, I’d rather go with something generic (and unfortunately less attractive than the LaCie boxes which are really nice) like the StorPack S35 as the unit for moving data around in.
The other major issue that I see is that we’re dealing with potentially sensitive data being moved around. I would seriously consider encrypting the data bricks that are rotated across sites since they are exposed to a higher risk of loss. (even from the Sheriff’s office)
I think that going this route protects you against any of the potential support issues that you’re more likely to see with commodity NAS and RAID boxes. Also there’s a much simpler expansion plan for dealing with archival data, either by expanding existing pools, or by using a data brick as an independent unit.
And I agree entirely on the comment regarding the home. The digital home looks more and more like a small business in terms of data management requirements. Worse, in many cases the small home has massively more data than many companies as it’s all media content. Considerably cooler data, but much higher volume. I just did a tour of the options available and ended up building a system much like the one described, with the exception that I plan to try and use ZFS send/receive over ssh to implement the offsite-component of the equation.
Interesting place, does he shoe horses too.
In your article you sound a little shocked. Don’t be. This is how the rest of the world is. Resellers are happy to make the pennies they do. Many of them work hard for their businesses.
A few years ago helping out a friend making site inspections for bids, I’ve been to large municipal transit operation centers. You will commonly see mission critical applications running on a low cost 5 year old pc, a ups and battery purchased with the system, and its used as a cup holder for the coffee machine. The system is backup one a month to a usb hard disk. The IT guy cleaned buses in his “spare time”. I been to several, and they are all basically the same. Each with dozens of vertical systems with independent service contracts from the sw vendor. When I asked “why don’t you use a lan based backup?”, the response from his boss was “what for, he does his job good enough with what he got!”.
Its a hard business..
On the other hand, some companies know how to play.
Like your sheriff buddy, sometimes the DOT or the Mayor will show up with grant money and an RFP or RFQ goes out.
One group purchased a bus driver scheduling program with the intent of replacing the old system. When the old scheduling software company realized they lost the bid, they promptly informed IT manager that the transit schedule information in the old system was considered property of the sw company and could not be transferred to the new system . Replicating the schedule on another system was considered copyright infringement. So, the new system sat there unused. The old system ? they are still paying 30% maintenance for the scheduling software.
Its a hard life out there
xfer_rdy,
I sincerely hope you recommended that your transit client review their contract with the sw vendor. No judge in any county of the US is going to uphold that the uniquely generated schedule information for a transit company is ever the property of the vendor supplying its software. It would be the equivalent of a company such as Intuit laying claim to your financial data or Microsoft laying claim to the content of your Excel spreadsheets.
The vendor can patent the code used to generate and store content, but in most (if not all) cases not the content itself. In fact, I don’t believe copyright law even applies to this scenario. And if the client happened to unwittingly sign itself into such a contract, that’s an unfortunate failure of the person who negotiated it in the first place. It makes me wonder about how frequently this bogus lock-in is exploited by vendors.
Here’s a simple solution:
3-node, 6TB storage cluster (standard x86 server hardware)
1 file serving node to support CIFS/NFS/FTP
Total for above around $18,000
1 16-port GigE switch and RJ45 cables ~$500
Total cost I don’t expect would exceed $19,000 (give or take and certainly not more than $20K). What do you get besides affordable storage:
– Simple management and administration
– Self-managing and self-healing storage cluster, set and forget
– Investment protection, seamlessly upgrade cluster nodes (HW) without disruption to data availability and accessibility
– Upgrade with newest in commodity server hardware you choose based on your price/performance requirements
– Long-term archiving of digital content without need for backup
– Assured data integrity and time-based retention
– Scale capacity as needed by simply booting a new node into the cluster
Voila! Simple, affordable, reliable storage.
It may not be the cheapest solution at acquisition, but if the customer is really worried about overall TCO and not getting nickled and dimed down the road, this solution addresses those concerns. It’s also ideally suited for unstructured data, e.g. still and motion images, like what he’s dealing with. Just throwing my 2 cents out to see if I can help. Let me know if you want to know more.
Scary, so far no single complete Solution.
How easy was it in old days, a backup software and a DAT drive, a proper backup schedule and a rotation plan for the tapes.
Where’s the easy straight forward solution today?
Simple. Buy a single Fireproof/Waterproof USB based ioSafe unit for each system from http://www.klsecurity.com (we do not sell these). These are great units for single box systems and have a patented fireproof /waterproof shell. Click on the Fireproof and Waterproof Data Backup and Recovery tab for all the options.
He can get a 1.5TB unit for $299. Heck, he could buy 2 for each system and just tie them together as a single disk.
Plug ’em in and go…
Hello Everyone. Here’s my proposal.
I am making this proposal based on the follow assumptions or observations.
1. The user is wary of establishing / maintaining N/W’s ( Though my proposal still advocates simple N/W setup )
2. The site disaster strategy is currently to stack data from the primary site to the secondary site on removable media, no replicated setup.
The proposal.
HP StorageWorks D2D2503i Backup System / HP StorageWorks D2D4004i Backup System
D2D2503i = 3TB of RAW capacity, 2 ISCSI Interfaces. / 1U Rack Mount /75MBs
D2D4004i = 4TB of RAW cap ( exp up to 9TB ), 2 ISCSI Interfances / 2U Rack /90MBs
These products also includes Includes Dynamic deduplication and low bandwidth replication. Thereby providing greater storage densities and longer data retention, which is clearly a value add though it looks more of a secondary interest.
The customer could go with a EH854A#ABA HP StorageWorks LTO-4 Ultrium 1840 SCSI External Tape Drive [Add $4,099.00] + HP Single Channel U320 PCI-Express HBA – Low Profile [Add $209.00]
For a D2D2T. Archival strategy to support System level / Site Level disaster recovery.
All this Totals upto System price $12,269.001
At this point the customer could hook up his two systems on Two unique Ethernet Cross cable so as to minimize the N/W spoofing fears. In the future he could use a dedicated switch to expand upto six servers which is what Backup System supports
Hey Robin,
A client of ours recently used the Mac OS version of our MatrixStore product to create a 12.5TB (usable, 2 copies of data. Always) disk based archive for US$ 16,000.
Blog post of how he did it here:
http://www.matrixstore.net/2009/03/18/matrixstore-assists-digital-production-workflows-in-brazil/
The benefit to the customer is that they get to work on the core competencies that add value to the business and not waste cycles on managing storage.
Ta
Nick