Who, what, where, when and why are the five Ws of journalism, authoring and research. But they’re the same fundamental questions swirling about artificial intelligence (AI). How your AI formulated the decision it made is a question AI researchers are racing to address.
Whether it is ChatGPT hallucinations, or citing sources involved in the answer provided, AI researchers, engineers and architects are looking at ways to address a key gap in the AI field. With more businesses and public organizations looking at AI to provide answers at scale and run key systems, they’ll need the accountability answering these five Ws provides. Without that clarity, trust for any AI or ML (machine learning) application or system will be increasingly hard to come by.
A key aspect of addressing the question overall is to know what happened to your data, when it happened, who made those decisions, where it happened and why changes or classifications were made to the data.
Auditability, traceability and lineage are key aspects of any data governance system. Even more so when it comes to data used in AI, and how AI uses it to make decisions. Customers often see Progress MarkLogic as a key aspect of their data management and governance strategy, due in large part to the 2015 inclusion of bitemporality.
Added in version 8, bitemporality is a feature allowing users to track both the valid time (when data is true in the real world) and transaction time (when data was entered into the database) for each record. This feature enables users to manage temporal data more effectively, making it easier to track changes over time and manage large volumes of changes in the data platform itself without the need for additional technologies in your architecture. This simplifies your tech stack and reduces overall system cost.
Combine this with MarkLogic’s rules-based search, and the Semaphore by Progress rule-based classification engine means that changes to the data—why it has been classified in such a way, what further context has been added and why certain data is more relevant to a query—and suddenly understanding human or AI-based decisions downstream becomes a lot easier to explain. Confidence in such decisions grows, and the ability to explain the rationale behind the decision becomes much easier.
Additionally, you can include geospatial metadata (another core feature in the MarkLogic Data Platform) to answer the where questions, as well as role-based access which clarifies data’s where and who.
Put all of this together, and you go a long way to answering the how of AI decision-making. I cannot tell you what goes on inside AI. And it will eventually get to the state where the decision-making process inside the algorithm or model is so complex no one can. Perhaps we’re even at that point in some cases now. However, we can answer the five Ws of your data right now. You simply need to use the MarkLogic Data Platform in conjunction with Semaphore Semantic AI technology. Together they provide a fully auditable, scalable and rule-based data management, classification and governance data platform designed with the future of data—AI or otherwise—in mind.
This all means that you have a robust, transparent, secure and agile data platform underpinning the builds of enterprise-ready, business-critical applications. This dual capability is critical for the future of AI in the enterprise and large public sector space. It is a daunting time, with so much change on the horizon. But with fully integrated Progress and MarkLogic offerings, you gain a trusted provider of the best products to develop, deploy and manage high-impact applications, like AI and ML solutions, ChatGPT, an automated AI agent or something completely new. Once you understand the five Ws and trust your data, the possibilities are truly endless.
This article came to mind after several customers, partners and prospects all expressed a keen interest in bitemporality, data lineage and data auditing, especially when discussing AI and ML applications. If you have another topic you would like me to discuss, a feature you would like me to explore or a use case you would like me to cover, then why not reach out. You can reach me at firstname.lastname@example.org or if you would prefer to contact me on LinkedIn, you can do so at https://www.linkedin.com/in/philipdavidmiller/. I’m always happy to hear from customers, partners and future MarkLogic and Progress innovators.
Philip Miller is a Customer Success Manager for Progress | MarkLogic, looking after our International Standards Bodies and Publishing accounts. Philip also leads our customer webinar series Digital Acceleration and Progress | MarkLogic Vision events. He’s always keen to advocate for his customers, and provide a voice internally to improve and innovate the Progress | MarkLogic Data Platform. He was named as a Top Influencer in Onalytica's Who's Who in Data Management. Outside of work, he's a father to two daughters, a fan of dogs and an avid learner, trying to learn something new every day.
Let our experts teach you how to use Sitefinity's best-in-class features to deliver compelling digital experiences.Learn More
Subscribe to get all the news, info and tutorials you need to build better business apps and sites
You can also ask us not to share your Personal Information to third parties here: Do Not Sell or Share My Info
We see that you have already chosen to receive marketing materials from us. If you wish to change this at any time you may do so by clicking here.
Thank you for your continued interest in Progress. Based on either your previous activity on our websites or our ongoing relationship, we will keep you updated on our products, solutions, services, company news and events. If you decide that you want to be removed from our mailing lists at any time, you can change your contact preferences by clicking here.