Another scrappy startup bites the dust
HP announced this morning that it is buying IBRIX, one of the up and comers for scale-out cluster file storage. They expect to close the transaction in 30 days.
The IBRIX secret sauce
IBRIX was founded in 2000 with IP from Yale math professor, Stephen Orszag. The essential insight behind the IBRIX architecture is the use of an “area code” to enable any node in the cluster to access a file in one hop. This eliminates much of the back-end chatter needed to maintain more granular cache-coherence in many clusters.
Like the Google File System, the IBRIX software is a layer on top of each node’s local OS and file system. It runs on Windows, Linux and – at least in the lab – Solaris.
The StorageMojo take
With Polyserve (transactional cluster storage), LeftHand Networks (cluster block storage) and now IBRIX (scale out cluster file storage), HP owns most of the prime real estate in the cluster storage market. As they note in their press release
Adding IBRIX’s software to HP’s portfolio further solidifies the company’s leadership in the emerging market of scale-out and high-performance computing storage, cloud storage, and fixed content archiving. This market is growing at a compound annual growth rate of 20 percent per year, which is faster than both the network-attached storage and total external storage markets.
What is surprising is that the other IBRIX resellers – EMC and Dell – didn’t make this move themselves. Rumor has it that IBRIX has been cash-flow positive with good growth in sales and customers over the last several quarters. Marketing myopia?
Courteous comments welcome, of course. I’ve done some work for IBRIX and like the technology and the people.
Wow, impressive move by HP!
2 years ago we have been talking to various vendors (HP w/ Polyserve, ONStor, Dell-EMC w/ IBrix, Isilon) for enterprise storage with “single/global” namespace.
Dell didn’t make the cut with IBrix because of them being separate vendors, so I always through EMC or Dell would buy IBrix in the end.
Not so much apparently.
However, this acquisition still does not fix the fact that HP has a bunch of hardware that they slap SW on, but the HW config just does not allow for the full potential to come out.
Example of bandwidth/throughput: an HP EVA with 100 FC drives could theoretically yield somewhere between 5 and 8 GB/s throughput (if you count per HD and add all up).
But because of the config and parts used (e.g. just 2 or 4 controllers), the EVA can’t even get to a 10th of that.
Same goes for MSA setup with a bunch of commodity SATA drives. Many drives, but the use of just 2 controllers just keeps the performance too low.
So this weak point can’t really be solved by Polyserve or IBrix, can it?
Disclosure: we bought EVA & MSA storage in 2007 and moved to Isilon IQ early this year. We still have & use he HP gear.
Anyone have any idea how much revenue IBRIX has and the price paid?
@Anonymous
Hmmm… methinks this is not well architected for performance.
JackRabbit 5U (JR5) 96TB unit, with 8 threads writing to the same file (each one writing to a different section of the file to reduce contention).
write performance below.
[root@jr5 ~]# mpirun -np 8 ./io-bm.exe -n 128 -f /data/file -w -s -d -v
N=128 gigabytes will be written in total
each thread will output 16.000 gigabytes
page size … 4096 bytes
number of elements per buffer … 2097152
number of buffers per file … 1024
[tid=1] file name = /data/file
[tid=3] file name = /data/file
[tid=4] file name = /data/file
[tid=0] file name = /data/file
[tid=5] file name = /data/file
[tid=6] file name = /data/file
[tid=7] file name = /data/file
[tid=2] file name = /data/file
Thread=1: time = 51.819s IO bandwidth = 316.179 MB/s
Thread=2: time = 51.824s IO bandwidth = 316.149 MB/s
Thread=3: time = 51.828s IO bandwidth = 316.123 MB/s
Thread=4: time = 51.832s IO bandwidth = 316.099 MB/s
Thread=5: time = 51.837s IO bandwidth = 316.065 MB/s
Thread=6: time = 51.843s IO bandwidth = 316.033 MB/s
Thread=7: time = 51.848s IO bandwidth = 315.999 MB/s
Thread=0: time = 51.848s IO bandwidth = 315.999 MB/s
Naive linear bandwidth summation = 2528.646 MB/s
More precise calculation of Bandwidth = 2527.989 MB/s
For laughs, lets write 256GB to the same file as well
[root@jr5 ~]# mpirun -np 8 ./io-bm.exe -n 256 -f /data/file -w -d -s -v
N=256 gigabytes will be written in total
each thread will output 32.000 gigabytes
page size … 4096 bytes
number of elements per buffer … 2097152
number of buffers per file … 2048
[tid=0] file name = /data/file
[tid=5] file name = /data/file
[tid=1] file name = /data/file
[tid=2] file name = /data/file
[tid=7] file name = /data/file
[tid=6] file name = /data/file
[tid=3] file name = /data/file
[tid=4] file name = /data/file
Thread=1: time = 102.272s IO bandwidth = 320.402 MB/s
Thread=2: time = 102.276s IO bandwidth = 320.389 MB/s
Thread=3: time = 102.281s IO bandwidth = 320.372 MB/s
Thread=4: time = 102.220s IO bandwidth = 320.563 MB/s
Thread=5: time = 102.297s IO bandwidth = 320.321 MB/s
Thread=6: time = 102.294s IO bandwidth = 320.330 MB/s
Thread=0: time = 102.307s IO bandwidth = 320.290 MB/s
Thread=7: time = 102.299s IO bandwidth = 320.317 MB/s
Naive linear bandwidth summation = 2562.983 MB/s
More precise calculation of Bandwidth = 2562.321 MB/s
Memory on that machine is 72GB, so both of these are well outside cache. This is a single file system. We present this out using NFS and others (GlusterFS, PVFS2, Lustre, …)
FWIW, these are 48 SATA drives.
Curious to know how many patents HP has from all of these purchases.
Kind of ticked off about this….HP buries good technology with bad hardware, and then doesn’t reveal pricing or allow interoperability with non hp solutions.
@ joe landman
thx for the extensive explanation. checked the product and it is quite different performance-wise than the HP stuff have (ie impressive).
However, we are currently running a full rack of Isilon IQ12000 nodes and the linear performance growth and the even better nett storage scalability (we grew for 3 nodes to full rack in half a year) is very satisfying to us.
A very important pro with Isilon: scaling of a single share or even folder up to the max of the cluster (up to the claimed max of 5PB). Who else can do that? With HP we’re managing various 16TB shares and fidgety NAS nodes (plain Proliant servers)… Too much headache!
Not rumor- fact. Ibrix was on fumes, and couldn’t raise any capital.
Buy price was less than what vc’s put into the company.
8 year old company with 150 customers?…So that means probably 50.
Curious to see how hp puts this acquisition into the stack.
Mostly because they never bothered to tap into a lot of markets that they could have dominated in. I was looking into a 750 TB solution tied together with IBRIX, but we’ll see now.
I mean seriously HP should be beaten for still selling a 2 Gb fibre storage array, as a current product.
HP’s track record is not so good lately.
Example: They purchased PolyServe and market it totally a single-use application (MS SQL Server), when Polyserve before the HP purchase had a good clustering/replication solution for Linux as well….but no where is it to be found now.
Ibrix was bust, no question about it, the rumour is that VCs are getting less than a $1 on $1.
No surprise there and correction for Robin, Eric Jackson was co-founder and invented everything, Steve Orszag has put his name and the story ends there.
Ibrix’s technology is ready, but, they didn’t do well bring their product to market. Either Dell, EMC, HP, IBM, even Cisco who all have proven capability bring product to massive customers.
So, whoever get Ibrix now, get cloud storage, HPC, and traditional/new storage era cutting edge.
Great Move for HP, it is a win win for HP and Ibrix.
1st EMC, 2nd IBM, 3rd HP, 4th DELL may change soon……..stay tune…..
IDEC, I agree with you, HP is completing the last few pieces of the puzzle in their storage picture. But, don’t forget, EMC has ATMOS, IBM has GFS, Dell seems doesn’t care about Ibrix kind of technology, and Cisco, sorry to say, an early player.
May not yet only change the order – refer to IDEC’s last line of his/her comments – 1st EMC, 2nd…..
In other words, instead, who can compete to stay in the top list? Who could get yanked out by lack of technology?? EMC is going to use their ATMOS to compete??? IBM uses their GFS? Symantec uses their SFS? NetAPP uses their ongoing project? HP is hopping fairly fast now uses their cheap token pick of Ibrix? And, Oracle? CSCO?
If the existing player does not sense or does not act to the rapid change of the storage diagram. Whoever is ready should be able to go first pick the top list – the $20 Bln market.
Echo to IDEC, as Arthur Cole mentioned in his article –
“…. at the same time…. ” (HP) “… don’t want to hand over a key enabling technology to your chief competitors.”
….(HP)…”would more likely use Ibrix to set up multi-petabyte architectures for top-tier customers.”
“…HP looks to have the three pillars of data center infrastructure well in hand.” PolyServ, Lefthand, Ibrix.
how much did ibrix sell for?
No idea. The rumor mill says “not very much” but I’ve yet to hear anything I’d consider authoritative.