The American Chemical Society worked with Datavid to replace fragmented legacy repositories with a cloud-based Content Lake designed to make decades of scientific publishing content easier to search, reuse, mine and use in downstream AI applications.
ACS had accumulated scientific books, posters, partner archives, conference video, marketing material and research datasets across different systems and formats. This made content difficult to discover and reuse, limited advanced search and analytics, and created barriers to text and data mining and future AI services.
Datavid built a central repository for both structured and unstructured material using Progress MarkLogic, Apache NiFi, AWS and Progress Semaphore. The system ingests XML, PDF, image, video and other formats. Transformation pipelines convert material into standard formats including JATS, while Semaphore adds semantic enrichment and links between scientific concepts. REST APIs and text and data mining query tools expose the resulting content to downstream services.
The project was designed as more than a cloud migration. The aim was to turn historically fragmented publishing assets into normalised, semantically enriched, machine-readable content that could support text and data mining, search, analytics and AI applications.
Benefits and outcomes
- More than 2,400 scientific posters were digitised and made searchable.
- More than 890 scientific books were processed for analysis and reuse.
- Datavid reports 50% faster data processing after the transformation.
- Datavid reports a 30% reduction in storage and management costs after migration to AWS.
- Automated transformation into JATS and other standard formats improved machine-readability.
- Semantic enrichment created a foundation for more sophisticated scientific search, linking and later AI services.
- The Content Lake provided reusable infrastructure for future digital products rather than serving as a single-purpose repository.
Datavid describes the work as making ACS content “ready for AI, text, and data mining.”
Mitchell Bakos, ACS Director of Product Development & Solutions Architecture, said Datavid’s data-management expertise had “truly set them apart.”
The performance and cost figures are reported by Datavid in its customer case study. They do not appear to have been independently evaluated.
Publisher / company: American Chemical Society / Datavid
Source: Datavid, From disconnected repositories to a unified content lake: Modernizing scientific content