US Air Force Research Laboratory Creates Unified Data Platform with Progress MarkLogic, Speeding Research Efforts
We used MongoDB early on, but we realized there are so many things that you can’t do with it, like built-in search, Semantics, and true RDF expression of our data models.
Matthew Jacobsen
Systems Analyst & Developer at US Air Force Research Lab
AI Summary
The challenge
AFRL research was hampered by a network of disparate data sources, inflexible software and lack of governance that limited research due to the lack of the ability to share knowledge and trace the pedigree of that knowledge.
The solution
AFRL deployed MarkLogic to build a unified data platform to help securely enable discovery and sharing of information.
The result
HyperThought™ eliminates many of the silos and reduces costs while easily discoverable historical tests improve efficiency across R&D.
The challenge
The U.S. Air Force Research Lab (AFRL), the primary scientific research and development center for the Department of the Air Force, is made up of about 700 in-house researchers, supported by thousands of external researchers around the world.
Collaboration was hampered as the external researchers worked with disparate databases and tools, resulting in numerous data silos that slowed critical research and decision making. AFRL was also challenged with massive data volumes, data velocity and variety. For example, a single simulation could produce an exabyte of data in one month and the organization catalogs more than 1.5 million types of metal alloys.
The disparate data sources, inflexible software and lack of governance limited research due to the lack of the ability to share knowledge and trace the pedigree of that knowledge. What was needed was a multi-model flexible data source that would enable AFRL to add search, security, semantics and performance at scale.
The solution
AFRL named the project HyperThought™. AFRL was determined that HyperThought be iteratively developed with agile principles, and be flexible, scalable and deployable with hybrid storage and connection models.
Early efforts with MongoDB showed that HyperThought held promise, but required a major effort to tether together critical missing pieces and derive benefit from the data. AFRL’s key priorities, search, security, semantics and performance at scale with a multi-model format were out of reach with that solution.
During the due diligence process, AFRL discovered the Progress® MarkLogic® data platform and determined it was the solution that could provide the multi-model context to cut through and unite the many disparate data sources to help speed research and promote smarter decisions faster.
The result
Combining best-of-breed commercial and open-sources technologies, HyperThought is providing AFRL researchers quicker access to data that is growing in complexity and helping them gain new insights into the data. It is a solution with two main pillars—an integration layer and systems connection layer. End users benefit from HyperThought’s extensive toolset, including content management, a digital workspace, equipment integration, workflow management and visualization. In fact, AFRL has realized a performance increase of 100X.
The MarkLogic-based platform delivers:
- Flexibility, Agility, Scalability - Adding new data sources, changing data models and increasing data volumes are now surmountable challenges with HyperThought.
- Efficiency - Historical tests and results are now discoverable and usable, improving efficiency across development and research.
- Collaboration - Scientists can now securely share data, which fosters collaboration with other researchers like Lawrence Livermore National Labs.
