It's frustrating, because a configuration like that makes the numbers completely irrelevant, unless you don't care whether your data gets persisted or not. I'd much rather see a more like-for-like comparison, and perhaps an unsafe configuration included for interest's sake.
Nobody in their right mind hits a key value store for each single key value (an RPC for each key value). You'd be limited by the network latency.
You batch your changes in the client and then you commit them to the server. When that commit succeeds your data is persisted; until such time any client application must consider the data uncommitted. This is not unlike a client application would deal with transactions to a relational database.
So the benchmark is completely valid here.
(Well, only that here we also have deferred log flush, which means the changes will take a second or two to actually make it to the file system, but that still demands the same aggregate throughput from the HBase. That is where this test is not fair).
> (Well, only that here we also have deferred log flush, which means the changes will take a second or two to actually make it to the file system, but that still demands the same aggregate throughput from the HBase. That is where this test is not fair).
That was actually the bit that I was concerned about - I failed to read the parent comment properly and assumed it had the exact same concern I did. If you don't have to do a disk write because you're not guaranteeing durability, you have a completely unfair comparison.
More accurately, the database was configured to buffer all writes until flush() was called, so durability was not guaranteed like it was on the other database instances. Still an unfair comparison, of course - I was just curious what you meant by 'not actually hitting the database', so I shared the tl;dr.
This isn't like mongodb's non-durable-by-default writes. No, the "buffering" here is done client-side. So it really wasn't hitting the database at all (until the client teardown phase, well after the latency that is being graphed here was measured).
cleanup calls flushCommits. So all requests are there. No get requests are served from the client's buffers so all of those must hit the region server. In addition the client side write buffer size is 12mb. So we're not talking about all requests being buffered at a time.
This may or may not be what is needed for your application so autoFlush is a per-client setting (HBase actually defaults it to on). Deferred log flushing is similar, though it's a per table setting.
Average latency is NOT correct because YCSB latency is measured per-insert, not averaged over the total runtime. Here is the code from DBWrapper.insert:
long st=System.nanoTime();
int res=_db.insert(table,key,values);
long en=System.nanoTime();
_measurements.measure("INSERT",(int)((en-st)/1000));
... where db.insert makes the htable.put call we're talking about.
The HBase default is correct for what YCSB is trying to measure ("live" updates, i.e., the update should be readable from the regionserver afterwards), but someone submitted a YCSB patch to override the default to make latency look better time some time ago, and most benchmarks including this one have inherited the mistake.
The client will block after the buffer is full, and will wait until the outstanding requests are ack'd from the server. So the times will still be pretty close to correct. Only the last few puts won't be timed.
