Why MarkLogic Won the Best Data Lineage Solution Award

Why MarkLogic Won the Best Data Lineage Solution Award

Posted on December 10, 2018 0 Comments

I’m delighted to write that on 7th December 2018, MarkLogic was recognized as the Best Data Lineage Solution in the 2018 Data Management Review Awards. It’s a real honour to receive this award, and it also presents an opportunity to explain data lineage, its use in financial services and why MarkLogic® is indeed the Best Data Lineage Solution.

The Importance of Data Lineage

Data lineage is about tracing the lifecycle of data – where it originates from, the path it takes and the transformations it undergoes. It’s all providing visibility and ultimately about whether data can be trusted. It’s a critical component of any effective data architecture.

Financial services firms have been having to deal with a plethora of new regulations over the last 10 years. Although the costs of complying with new regulations may have peaked, updates to existing and new regulations will continue to come into force. Understanding the lineage of trades, models, client communications, payments, etc. is vital to not only staying compliant but staying compliant effectively. Too often, when the phone call from the regulator comes, organizations struggle to put together the information needed. This process always involves data lineage – tracing back from symptoms to cause. With data lineage built into a data architecture, this process is accelerated and more accurate.

MarkLogic and Data Lineage

Data lineage is built into MarkLogic from the ground up. Original data is imported “as is” and can then be progressively transformed using “envelope” patterns to enable and maintain data lineage. The original data is stored, and transformations are kept as metadata. The entire lifecycle of every piece of data can therefore be replayed and examined.

This process avoids the brittle process of ETL (Extract-Transform-Load) that is a characteristic of conventional relational systems, and that causes such difficulties when understanding data lineage. For example, was a specific data value part of the original record, or was it added during a subsequent ETL step?

Data Lineage in Action

Many of the largest global investment banks use MarkLogic for post-trade processing and regulatory reporting, and one case in particular helped MarkLogic win its category in the 2018 Data Management Review Awards. ABN Amro selected MarkLogic because of its ability to integrate complex trade data across asset classes and several trading systems quickly and cost-effectively.

ABN Amro needed a database that could provide a fully integrated, consolidated view of data to provide a single source of truth for reporting purposes as well as to provide alerts when behaviour is outside normal parameters. MarkLogic filled the bill because its fast development could support the bank’s timelines. Additionally, MarkLogic’s in-built lineage capabilities allows ABN Amro to minimize risk through “tech time travel”– rewinding the database and examining trade lineage. Furthermore, ABN Amro can more easily comply with future regulations as all required data is in one place, which provides a uniform and consistent view of orders and trades.

As Jaap Boersma, Principal Architect for Capital Markets Solutions, ABN Amro, said, “We need the ability to respond quickly to changing regulatory requirements. We chose MarkLogic because trade data is notoriously difficult to handle in relational databases. We found the breadth of MarkLogic’s multi-model database and associated features, such as security and ACID transactions, to be compelling.”

Thank You!

We at MarkLogic have worked hard to create a trusted, enterprise-class NoSQL database with data lineage built into its core. We are grateful that our revolutionary approach to Big Data challenges has helped so many of our customers. We are proud to be recognized as the Best Data Lineage Solution by the 2018 Data Management Review Awards.

To find out how we can help your organization build a trade data hub, please visit our website, or contact us by sending an email to tradedatahub@marklogic.com.

Giles Nelson

Giles is currently Chief Technology Officer for Financial Services at MarkLogic where he’s responsible for MarkLogic’s technology go-to-market in financial services worldwide.

Giles has had 20 years of experience in the software and financial services industry. He was previously at Software AG where he had global responsibility for its Big Data and Streaming Analytics business. Previous to that he was at Progress Software where he was instrumental in taking its innovative financial services products global. Giles joined Progress when it acquired the event processing and Streaming Analytics vendor Apama, a company he co-founded which acquired over 150 clients in financial services.

Giles holds a PhD in Computer Science from the University of Cambridge.


Comments are disabled in preview mode.

Sitefinity Training and Certification Now Available.

Let our experts teach you how to use Sitefinity's best-in-class features to deliver compelling digital experiences.

Learn More
Latest Stories
in Your Inbox

Subscribe to get all the news, info and tutorials you need to build better business apps and sites

Loading animation