Kdb+ is a time series database built for the Financial Services Industry to handle historical data used by 20 out of 21 top global banks. The STAC Antuco benchmark and STAC Kanaga benchmark test multiple aspects of database and HW performance on a typical customer workload. Both are IO-stressing benchmarks with various types of access (random-like, sequential like, with overlap, etc.)
"Our customers are at the forefront of high frequency sensor analytics, financial trading, streaming data surveillance, regulatory monitoring, and real time analytics. To stay competitive, they are in the chase for increased amounts of online memory in order to improve their “time to result” of time-series data analytics. Intel® Optane™ DC persistent memory provides our customers with the opportunity to achieve this by giving customers systems with a massively increased memory set. As both an in-memory and on disk database, we have always pushed against the limits of memory technology with kdb+. This recent testing with Storage over App Direct mode provided access to significantly larger sets of kdb+ in-memory data. Intel® Optane™ DC persistent memory truly enlarges the capabilities of Kx technology by taking full advantage of its unique architecture."
-Mark Sykes, CTO of First Derivatives and COO, Kx
Challenges
Quick access to historical data is absolutely critical to electronic trading to get a competitive advantage. The dataset size is often too large to be stored in DRAM. Adding additional memory is often cost prohibitive, resulting in the need to scale out and increasing HW footprint.
Solution
Higher memory capacities of Intel® Optane™ DC persistent memory (App Direct Mode (Storage over AD)) facilitates storing larger datasets and more historical data closer to the CPU, delivering significant performance improvements (see charts)1 2 3 4, compared to using SSDs (current solution).
Value
Significantly better performance – Kx customers will benefit from being able to access historical data much faster than currently available solutions.