Nasdaq case Study

For more information about other Microsoft customer successes, please visit:

www.microsoft.com/casestudies

Customer: NASDAQ OMX Group

Website: www.nasdaqomx.com

Customer Size: 2,433 employees

Country or Region: United States

Industry: Financial services—Capital markets and securities
Customer Profile

National Association of Securities Dealers Automated Quotation (NASDAQ) OMX Group is a leading provider of trading, exchange technology, information, and public company services across six continents. Based in New York City, the company earned US$1.9 billion in revenue in 2013.

Software and Services

Microsoft Server Product Portfolio

Windows Server 2008 R2 Enterprise

Microsoft SQL Server 2014 Enterprise

				NASDAQ OMX Group Reduces 2 PB of Data to 500 TB with Microsoft In-Memory Technology

“SQL Server 2014 is a game changer for us. Its in-memory columnstore, advanced statistics, and cardinality estimation are the tools we need to manage our very large databases for the long term.“

Stanley Orlowski, Director of Database Structures, NASDAQ OMX

NASDAQ OMX Group is a leading global provider of financial trading services, exchange technology, and market insight. Every year, it adds more than 500 terabytes (TB) of uncompressed data to its archives. To better manage data growth, maintain fast queries, and reduce database maintenance time, NASDAQ OMX migrated data warehouses to SQL Server 2014. As a result, the company will save 500 TB of disk space this year by using new data compression and indexing capabilities. The company is also saving time and money on data management, improving query performance, and helping to facilitate long-term goals.

Business Needs

Established in 1971 as the world’s first electronic stock market, NASDAQ OMX owns and operates three clearing houses, five central securities depositories, and 26 markets (including the NASDAQ Stock Market) with a combined value that exceeds US$8 trillion. NASDAQ OMX is also a technology company. Its infrastructure includes custom applications and a trading engine that’s used internally and by 80 global marketplaces.

The volume of transactions and data that NASDAQ OMX manages is impressive. When markets open, the company processes more than 1 million messages per second. In addition, the company manages many very large databases. Stanley Orlowski, Director of Database Structures at NASDAQ OMX, says, “Just our US Options and Equity data archive handles billions of transactions per day, stores multiple petabytes of online data, and has tables that contain quintillions of records about business transactions.” Uncompressed, the Options and Equity archive measures 2 petabytes (PB). With page compression, it measures 1.5 PB.
Financial traders need near-real-time insight into current and historical data to make decisions, in just seconds, that can involve millions of dollars. Analysts also need in-depth insight to understand trends and create regulatory reports. To meet these requirements in the past, IT staff had to constantly monitor and fine-tune databases so that they could provide the best possible performance, especially with uncompressed data growing by 500 TB each year.
By 2013, NASDAQ OMX recognized that its existing database solutions couldn’t support projected long-term growth. Orlowski says, “Data is the lifeblood of the company—and managing a petabyte of data takes a lot of planning. You can’t turn quickly on a dime.”
Solution

When NASDAQ OMX learned about SQL Server 2014 software and its in-memory columnstore, IT staff decided to migrate data warehouses to SQL Server 2014 Enterprise and the Windows Server 2008 R2 Enterprise operating system. By choosing SQL Server 2014 and taking advantage of in-memory columnstore—as well as other new capabilities—the company can better manage its massive amounts of data, speed queries, and simplify database maintenance.

Not only does in-memory columnstore serve as the only index for a table, but also it can increase data compression ratios, and it supports live table updates.
NASDAQ OMX can also improve query performance and simplify maintenance by taking advantage of incremental statistics and the changes in the cardinality estimator in SQL Server 2014. With these new tools, table statistics can be updated more frequently, and they can be updated for individual partitions rather than for an entire table. As a result, statistics are more accurate, which helps produce better query plans and faster query performance. SQL Server 2014 also supports native backup encryption so that engineers can encrypt data in one step rather than two—and encrypt directly to disk, tape, and the cloud.

Commenting on the company’s continued use of the Microsoft platform, Orlowski says, “The Microsoft technology stack gives us excellent total cost of ownership, and we have a high degree of confidence in the platform. We’ve also had SQL Server in place for well over a decade on mission-critical systems, and it has a positive track record for delivering reliable performance.”

In 2013, engineers initially migrated the Nordic OMX data warehouse (which includes 8 TB of page-compressed data) to SQL Server 2014. Engineers are currently upgrading the US Options and Equities archives to SQL Server 2014. They expect to complete the database upgrade by the middle of 2014 and finish converting tables to in-memory columnstore by the end of 2014.
Benefits

By moving its data warehouses to SQL Server 2014, NASDAQ OMX saves time, money, and disk space—and keeps its massive data volumes manageable.

Compresses Data to One-Quarter Its Original Size, Saving 500 TB of Space

In the past, NASDAQ OMX could compress 500 TB to 250 TB with page compression in SQL Server 2012. However, with SQL Server 2014, the company can realize even greater compression levels, reducing data to one-quarter of its original size. Orlowski explains, “With SQL Server 2014, we can reduce 250 TB of page-compressed data to 125 TB using in-memory columnstore. After we migrate all of our US data archives to SQL Server 2014, we expect to reduce our storage footprint by half a petabyte. We’ve seen on average a 50 percent reduction in the amount of data that’s physically stored on disk. And not only does it help us save money, but it also helps us more efficiently and effectively deliver data to our consumers.”

Speeds Queries and Increases Accuracy

NASDAQ OMX also expects to facilitate faster and more accurate insight into its massive data stores. “After an initial migration to SQL Server several years ago, we were able to rework queries so instead of taking a day or two to run, these same queries run in just minutes,” says Andrew Geaslin, Principal Database Administrator at NASDAQ OMX. “Upgrading databases to SQL 2014 and taking advantage of in-memory columnstore will make it easier for us to hold on to those performance improvements because statistics maintenance is easier.”

Saves Time Spent on Maintenance and Cuts Storage Footprint in Half

IT personnel will also save disk space with in-memory columnstore and new encryption capabilities. “SQL 2014 allows us to save disk space on multiple fronts, making it possible for us to cut our existing storage footprint for the Options and Equities archive in half,” says Geaslin. “The database itself has a smaller storage footprint on disk due to in-memory columnstore, and encrypted backups reduce the need for swap space on disk used by third-party tools to encrypt data before it’s moved to storage.” IT personnel will also increase efficiency and reduce complexity. “By consolidating multiple indexes into a single in-memory columnstore, we reduce our maintenance and troubleshooting overhead,” says Geaslin.

Improves Decision Making and Helps Facilitate Long-Term Goals

Maintaining its ability to deliver rapid and accurate insight into current and historical market data is critical for NASDAQ OMX. With its new solution, the organization has the platform it needs to effectively manage petabytes of information. “SQL Server 2014 is a game changer for us. Its in-memory columnstore, advanced statistics, and cardinality estimation are the tools we need to manage our very large databases for the long term,” says Orlowski. “By making data more readily available to our customers, we can also help them make better business decisions.”