by Michał Cukierman
7 min read
Stop loading backend data into the CMS
by Michał Cukierman
7 min read
First published on LinkedIn
This article was originally published on Michał's LinkedIn. We are republishing it here with subheadings added for clarity.
During my AEM development experience, I've learned a valuable lesson: loading all third-party data into the JCR can cause problems. Here's why.
Fig. 1: Batch data loading and its problems
Data loading into the AEM takes a long time, usually it has to be done using nightly batch jobs executed during off-peak hours. This often causes the information on a site to be outdated.
A large JCR repository slows down AEM, impacting both site rendering and content authoring. Jackrabbit, designed for content, struggles with massive data loads, causing lengthy maintenance tasks like backups and compaction.
Replicating large data volumes between Author and Publish instances can be slow, affecting content publishing workflows.
Point-to-point integrations and logic implemented on the Author instance complicate the codebase and make it harder to maintain.
A single monolithic Java application handles data loading, transformation, cleaning, and replication. This makes scaling difficult and creates a maintenance problems with all business logic tied to one CMS project.
Publish instances, responsible for delivering data to users, can become slow as the JCR grows.
These issues become more prominent as your data volume increases. Which is what you should be expecting.
Fig. 2: Batch data loading and its problems
Digital Experience Mesh lets you orchestrate data from all sources without loading it into AEM. Real-time data pipelines powered by microservices can process millions of updates per hour, ensuring your website reflects the latest information from backend systems.
Delivery Services like: OpenSearch, Nginx or REST endpoints created Unified data layer for your organization. The same data can be used by all the channels including sites created in different technologies like Edge Delivery Service or a frontend application.
This is what we call true composability. Move away from CMS-centric architectures and leverage AEM for its content management strengths. With a modern data architecture, your website will always deliver fresh, up-to-date data using the latest technologies. Adding or replacing a source system or delivery service can be done anytime without costly migrations.