April 08, 2016
Atlantis HyperScale testing with Jetstress
If you haven’t already done so, take a look at our latest work with Howard Marks of DeepStorage, LLC. We worked with Howard to test Jetstress against one of our Atlantis HyperScale appliances, namely a CX-12 unit with default configuration. What we wanted to achieve was to see how much storage I/O performance the HyperScale platform could theoretically drive with an Exchange workload.
You can get a hold of the Technology Validation Report from this link.
We performed our testing on a HyperScale CX-12 system, with 384GB of RAM per node. The system was configured with vSphere 5.5 and vCenter 5.5. The storage was configured as three vSphere datastores, each under a terabyte in size. As with all Atlantis datastores, the data reduction features cannot be turned off, therefore features such as inline dedupe are always on.
Three Jetstress servers (VMs) were then created and were distributed across the three Atlantis datastores across the four node CX-12. This system can, therefore, tolerate a node failure while still providing sufficient resources for the affected VMs to restart on the three remaining nodes. The system was configured to tolerate a one-node failure and the Jetstress environment and the HyperScale appliance were configured with the three datastores and the three Jetstress servers to accommodate this.
Each Jetstress server was provisioned with eight logical drives (VMDKs) in order to store the roughly 16TB of databases we needed. The system’s data reduction features were fully utilized to take advantage of the capacity by creating thin provisioned VMDKs approximately 800GB each.
Note that Jetstress data is significantly more efficient with data reduction technologies such as data deduplication than real world Exchange data. This of course has a significant impact on the amount of storage capacity consumed but on a one tier storage architecture such as Atlantis HyperScale, it does have no significant impact on performance due to our use of server memory for the majority of the data reduction processing.
The Microsoft Jetstress Field Guide documentation describes Jetstress as:
Jetstress is a tool for simulating Exchange database I/O load without requiring Exchange to be installed. It is primarily used to validate physical deployments against the theoretical design targets that were derived during the design phase. Jetstress testing provides the following benefits prior to deploying live users.
- Validates that the physical deployment is capable of meeting specific performance requirements
- Validates that the storage design is capable of meeting specific performance requirements
- Finds weak components prior to deploying in production
- Proves storage and I/O stability
Jetstress is a very easy tool to use, you only need to define the number of mailboxes, their size and the number of IOPS per mailbox. If the system can perform the I/O load specified, Jetstress will report a pass, if it can’t Jetstress reports gets a fail. It’s very binary and black and white. Either you win or you lose!
Jetstress is NOT Exchange
Jetstress uses the Jet database engine that is at the heart of the Exchange server. Jetstress performs read, insert, delete and other operations against the Jet database to simulate Exchange users. By accessing the core database directly, Jetstress can create much higher levels of disk I/O than using full Exchange servers that have the overhead of processing email before writing to the database.
But Jetstress is not Exchange. It doesn’t actually process email and as a result Jetstress uses significantly less CPU and memory than real Exchange servers. It would be more realistic to use full blown Exchange servers but the very complexity of creating an Exchange configuration large enough to stress a modern storage system is a big job and such a system would still need another tier of servers to emulate the thousands of users sending and reading email.
This very complexity is why Jetstress was created. So users could validate that a storage system could handle the demands of their Exchange estate before they actually build their Exchange estate.
Jetstress is just a storage benchmarking tool that simulates the Exchange Jet engine. The report is not intended as a reference architecture for building a real world Exchange environment. For help with designing an Exchange environment on HyperScale please contact Atlantis on help and guidance on sizing your Exchange deployment on HyperScale. The report is designed to test the capabilities of Atlantis HyperScale with Jetstress as the load generator.
The goal of Jetstress testing is to determine the maximum number of mailboxes the system can support over the two hours it takes to perform a valid Jetstress run. This requires multiple Jetstress runs with different parameters, as we iterate to the results that we’ll report.
Since performance testing takes two hours, and the checksum verification at the end of each test adds several additional hours, we would be lucky to get in even two runs a day if we were running Jetstress manually. To accelerate this process, and to save our own sanity, we developed a set of tools that automate the process.
The JetTest tools allow us to synchronize the start times for tests across multiple machines and more importantly, to queue up multiple tests with different numbers of mailboxes, IOPS/mailbox or execution threads and to run them automatically.
As we described above, Microsoft’s Jetstress uses the same Jet database engine to simulate some set of user mailboxes. Our tests targeted a configuration of:
- Three VMs running Jetstress 2013, each configured with
- Windows Server 2012 R2
- 8 vCPUs
- 64GB vRAM
- LSI Logic SAS controller
- 50Gb boot disk
- Eight 800GB thin provisioned VMDK for Jetstress databases
- Eight databases per server
- 60,000 mailboxes – 20,000 per Jetstress server
- With a large mailbox size of 800MB
- 0.121 IOPS per mailbox (150 messages per day plus 20% headroom)
- Background database maintenance enabled
- Two database copies
The three Jetstress servers were each run from a separate server node and each stored its data in an independent volume.
This configuration can tolerate one node failure without impacting performance, i.e., the failure to tolerate is one node (or one host failure).
The configuration targeted a total of 7260 IOPS (60,000 mailboxes X 0.121 IOPS/mailbox) with latency within Jetstress’ strict limits – most significantly, database read latency below 20ms – and achieved 8798 IOPS, or just over 20% more than required, while averaging 7.2ms of latency.
Aggregating the performance of our three Jetstress servers, we can see in the table below that the HyperScale system significantly exceeded our targets. We could have repeated the tests with 72,000 mailboxes, but we didn’t need to since 60,000 is such a nice round number that we thought it would make a better headline.
|Average database read latency
|Average log file write latency
As demonstrated, the theoretical number of mailboxes that a HyperScale CX-12 unit can achieve is 60,000 in a 4-node hyperconverged solution. This shows that HyperScale is more than capable in tolerating the I/O requirements of the Exchange Jet database.
This benchmark with Jetstress is just one of the tests that we are actively working on to validate the capabilities of HyperScale. In upcoming benchmarks, we have submitted a HyperScale appliance to StorageReview.com for their independent testing with OLTP workloads (Sysbench and SQL Server) and also VMmark benchmarks. Watch this space.
Current rating: 5 (7 ratings)