A Web-based News Portal Powered By Strapi and AWS

The client is one of the leading content creators across the USA and UK broadcasting other programs, including musical shows, quiz shows, interviews, and much more.

The project is to develop a platform for users to read news feeds and watch video content based on their specific region and the users can set preferences for live updates and content. 

Solutions for the platform is done keeping in mind the following requirements:

• Performance 
• Multi-device and multi-channel support
• Scalability
• Customizable

For achieving the above-mentioned objectives for the CMS we recommend the following components and engineering practices.

Headless CMS

Recommended platform:  Strapi Opensource headless CMS

Using a headless CMS will help us accommodate the flexibility of a completely custom site along with the benefit of easy content creation similar to traditional CMS.

Pros:

• Highly performant.
• Highly customizable and future proof
• Freedom to choose any client-side frameworks
• Easily supports multi-device content delivery.

Cons:

• More development effort compared to traditional solutions like WordPress since the client-side implementation will need more effort.

Horizontally scalable architecture

The system will be deployed in AWS following architecture best practices that support load balancing and auto-scaling which will help in handling traffic spikes in an efficient manner. 

All contents which can be pre-computed for faster client consumption will be pre-processed and stored in static storage like s3 and delivered via CDN for faster access. For example, all video uploaded will be encoded to an appropriate format which can be used for better streaming based on network conditions using adaptive bitrate streaming for the end-user.

Front-end assets will be served via AWS CloudFront CDN for better performance on the client side across different locations.

This enables the platform to scale on-demand and also be highly available.

Context Diagram

The system leverages three-tier architecture and uses external cloud-based services(AWS and elastic cloud) for the features in certain modules. The News App will contain web, API and datastore. 

Containers Diagram

This level shows the technical building blocks of the system.

• Client apps – Admin and user React apps using REST/Graphql APIs to communicate with the server. Apps get their static contents directly from AWS CloudFront CDN to retrieve data faster.
• API Layer – Handles all application business logic, Uses third-party services using its HTTP service or SDK to initiate calls. It combines two key methodologies for processing data and hydrating UI Layer – Headless CMS and Server Side Rendering.
• Datastores – AWS Aurora DB will act as the primary datastore for content and transactions. Data level caching will be handled using Redis and caching on edge for page contents will be handled using AWS Cloudfront CDN distribution.

API Layer Deployment

• Both Next.js and Headless CMS engines will be deployed within a single pod so that it’ll be treated as one unit in the deployment cycle and horizontal scale configurations.
• Maximum reduced latency as the containers are communicating within the pod in bridged networking mode.

Security

• JWT based Bearer token Authentication and Authorization.
• Strict Access Control logic built into the Data access layer.
• Option to create a role.
• Options to assign permissions to the created role: Create, Read, Update or Delete of any entity on Role level.

Layered structure & state

• The UI app will be architected with module-based layering so that we can easily enable lazy loading of modules reducing initial load time.
• Functional hook based modern react components will be used
• Sass based modern styling principles will be used
• The service layer within the application will guarantee separation of concerns and scale
• Isomorphic app with both client-side and server-side rendering.

Here is the datastore details.

Primary Database

AWS Aurora will be used as the primate DB to store all transactional and platform-level data.

Resilience & Disaster Recovery

The system will use Read/Write replicas to ensure availability. Using read replicas, you can elastically scale out beyond the capacity constraints of a single DB instance for read-heavy database workloads. You can promote a read replica to a standalone instance as a disaster recovery solution if the source DB instance fails.

Amazon Aurora features a distributed, fault-tolerant, self-healing storage system that auto-scales up to 128TB per database instance. It delivers high performance and availability with up to 15 low-latency read replicas, point-in-time recovery, continuous backup to Amazon S3, and replication across three Availability Zones.

Security

The database will be provisioned under the VPC subnet and a proper security group will be added to ensure no other services can access the same. VPN will be used for IAM Access as well.

Scalability –  Data Volume & Traffic

• Master-slave setup in Aurora with cluster mode.
• Amazon Aurora is designed to offer 99.99% availability, replicating 6 copies of your data across 3 Availability Zones and backing up your data continuously to Amazon S3. It transparently recovers from physical storage failures; instance failover typically takes less than 30 seconds. You can also backtrack within seconds to a previous point in time to recover from user errors. With Global Database, a single Aurora database can span multiple AWS Regions to enable fast local reads and quick disaster recovery.

Caching 

• All processed data will be stored inside Redis. Redis will run in cluster mode to ensure handling load and distribution of data.
• All processed content ready to serve by clients will be kept in Redis so that it can easily look up and served to clients.
• caching on edge for page contents will be handled using AWS Cloudfront CDN distribution and all processed HTML will be available there.

Search Engine

We’re using elastic search as the search engine and elastic cloud SaaS service will be used to host the service. We can leverage the ELK stack within the SaaS setup for monitoring indexes.

Elasticsearch also covers multi-language search requirements in the application.

Here is the workflow.

Request Cycle

Our user request cycle will have the following steps which are designed to maximize performance.

• User visits homepage
• The browser sends request to API Layer (Next.js)
• Next.js is configured for SSR and identifies that the request is for a whole page, not just a data payload.
• The server checks the cache for the data based on the request params and headers.
• On a cache miss, the Server generates the whole HTML using Strapi API for data.
• The client receives the HTML content and it gets rendered swiftly.
• Any static URLs mentioned in the HTML are delivered via CDN with minimum latency.
• The parallel browser downloads the bundle for react-specific modules in lazy loading style and hydrates the app on the client side.
• All subsequent requests will be API driven and data will be loaded asynchronously giving a faster user experience.

The deployment is orchestrated with Amazon Elastic Kubernetes Service (EKS) for end-to-end deployment workflow. We’ll have CI/CD pipelines running from the source repository to initiate deployment.

Pipelines will have continuous integration test suites to ensure a tight feedback loop for each deployment.

• All backend services will be deployed within VPC and will be unavailable for public access directly.
• Client to server communication will be protected with tokenization and HTTPS.
• API servers are exposed via Load balancer alone. All compute resources are behind load balancers and CDN.

Challenge 1: Text To Speech – Conversion

Requirement: The requirement for this feature was to make the news content available for users to listen to in the form of audio.

Solution: It was implemented with APIs and the Webhook feature of BeyondWords.io. The solution is that the content of the news will be automatically sent to beyondwords using their APIs on the publishing of the content. Once the processing of beyond words is completed, it will send back the information from which the audio can be retrieved and with the help of a normal audio player, it can be added to the news page also.

Challenge 2: Geo-tagging news (Local News)

Requirement: Tag news and make it available based on the region

Solution: Proposed solution for this requirement was to integrate the custom field feature of strapi recently released and to use google maps to tag the news while it is created. With this feature, the admin can create news with the latitude and longitude of the news area or the locality added, which the application can query accordingly.

Challenge 3: Dockerization of Strapi v4

Requirement: Dockerized deployment of Strapi v4 application

Solution: Dockerization of the strapi v4 application had a few technical issues as the official docker image of strapi only had support till 3.x. We created a custom one with a NodeJS base and it was not an optimized one that had breaking issues too. With research through the strapi documentation, we came across a community plugin recommended by strapi which makes the task easier. 

Challenge 4: AWS S3 Integration

Implementation : Uploading the images from the content editor and other sections

Solution: A plugin was used and with custom configurations done with the CKEditor config, we achieved the solution as required.

Challenge 5: Elastic search

Implementation: To create a unified and faster search feature

Solution: We used the cloud service of elastic search to create the indexes with HTTP API, which will work according to the CRUD operations of strapi. It was also integrated in a way to work perfectly with the draft & publish feature too. It was integrated into the front end which allows a user to search at once and retrieve results from News, Stories, and Videos.

Challenge 6: SSR

Requirement: Make the webpages search engine friendly and render properly.

Solution: The solution was simple. We implemented the SSR (Server Side Rendering) feature of the ReactJS framework NextJS. With this, we met the requirements of most of the functionalities required for SEO including the meta tags and dynamic content.

Next.js 

• To take care of React specific layout declarations, routing and server side rendering.
• Isomorphic by default, i.e it supports Server side rendering (SSR), client side rendering (CSR) and Static Site Generation (SSG). We can fluidly mix and match the techniques to get to a custom workflow that’s appropriate for the App.

Strapi

• Truly headless CMS with complete customization.
• We can leverage the built-in Strapi admin interface to create all news and article content.