← Back to blog

Unveiling Sided: Revolutionizing News Consumption with Media Bias Awareness in SL

May 2023Case study9.2K

Introduction

sided.news is designed to expose the tactics employed by biased media organizations and shed light on their hidden agendas. By aggregating news articles from various sources and contrasting their reporting styles, this platform equips users with the tools to discern the truth amidst a web of manipulation.

You can read more about this in this article Worst media behaviour in election history [1]

I believe in the power of knowledge and the importance of an informed electorate. sided.news serves as a centralized hub for news consumption, allowing users to compare and analyze how different media outlets portrayed the same events during the 2019 presidential election. By providing an unbiased and comprehensive perspective, this platform enables users to make informed decisions based on a more accurate understanding of the political landscape.

Through sided.news, users can develop critical thinking skills, challenge prevailing narratives, and cultivate media literacy. I aim to empower individuals to reclaim control over their information and navigate the media landscape with clarity and discernment.

Newer

Older

Requirement Gathering

To ensure the successful development and deployment of the sided.news application, thorough requirements gathering is essential. Here are the key aspects to consider:

  • Data Sources: Identify five media websites from which news articles will be pulled. Ensure these sources represent diverse perspectives and ideological backgrounds, allowing users to compare how different media outlets report the same news [2].
  • News Aggregation: Determine the method and frequency of aggregating news articles from the selected sources. Consider using APIs or web scraping techniques to retrieve the latest articles and keep the platform updated with fresh content.
  • User Interface: Define the desired user interface, focusing on a clean and intuitive design that enables users to easily navigate and switch between different media sources. Consider features such as search functionality, filters for sorting articles, and options for saving or bookmarking articles of interest.
  • Content Display: Determine how the news articles will be presented to users for comparison. Implement a side-by-side view, allowing users to read the same news story as reported by different media outlets. Highlight key differences in reporting styles, language, or emphasis between articles.
  • User Interaction: Provide users with the ability to switch between media sources and explore different perspectives on a particular news story. Allow users to leave comments, rate articles for credibility or bias, and share their own insights and observations. Encourage active participation and critical thinking.
  • Performance and Scalability: Consider the potential increase in users and news articles over time. Ensure the system can handle a growing user base and the continuous aggregation and display of articles without compromising performance.
  • Security and Privacy: Implement appropriate security measures to protect user data, including secure authentication and data encryption. Comply with privacy regulations and establish clear policies regarding data usage and storage.

Existing Systems

Currently there are no comprehensive platforms that offer the exact functionality and features provided by sided.news in SL. While there are various news aggregators, fact-checking websites [3], and platforms that analyze media bias, none of them specifically focus on comparing news articles from multiple media outlets in one centralized location.

System Architecture

Newer

Older

sided.news utilizes a powerful architecture combining Next.js, Node.js, and MongoDB to deliver an efficient and seamless news aggregation experience. Here's how it works:

When a user visits sided.news, the Next.js frontend sends a request to the Node.js backend API. The API is designed to check whether the specific route has been accessed within the last 10 minutes. If it has, the API retrieves the stored news articles directly from MongoDB, eliminating the need for scraping.

However, if the specific route has not been accessed recently, the backend employs the Cheerio [4] npm library to scrape the relevant websites associated with that route. By leveraging Cheerio, the backend can extract the desired news articles from the target websites effectively.

Once the news articles are scraped, the backend stores them in MongoDB, creating a cache for future requests. This approach significantly reduces the need for repetitive scraping, resulting in improved performance and reduced load on external websites.

The integration of Next.js as the frontend framework ensures a fast and responsive user experience, while Node.js powers the efficient backend API. MongoDB acts as the persistent data store, storing the scraped news articles for quick retrieval.

Overall, this architecture allows sided.news to provide users with up-to-date news articles while minimizing the scraping overhead. By combining Next.js, Node.js, and MongoDB, sided.news achieves a robust and scalable platform that seamlessly aggregates and delivers diverse news perspectives to its users.

Features

Multi-Source News Aggregation: sided.news pulls news articles from multiple media websites, allowing users to access a wide range of news sources in one centralized platform.

Newer

Older

Real-Time Scraping: The Node.js backend scrapes the websites associated with specific routes, ensuring that the latest news articles are available to users upon their visit.

Newer

Older

Caching Mechanism: Scraped news articles are stored in MongoDB, creating a cache that reduces the need for repetitive scraping and improves overall performance.

Media Bias Awareness: sided.news aims to raise awareness about media biases by presenting news articles from various sources, encouraging users to critically analyze and compare different perspectives.

Mobile-Friendly Design: sided.news is designed to be responsive and optimized for mobile devices, ensuring a seamless user experience across different screen sizes. Also the mobile app for the sided.news is also released and you can find it in play store

Route-Based News Retrieval: When a user accesses a specific route, sided.news checks if the news articles for that route exist in the cache. If available, the stored articles are fetched from MongoDB, providing a faster response time.

Efficient Frontend Framework: sided.news utilizes Next.js as the frontend framework, enabling fast rendering and responsive user interactions.

User-Friendly Interface: The app features a clean and intuitive user interface, allowing users to easily navigate between different media sources, compare news articles, and explore diverse perspectives.

Implementation

In the development of sided.news, creating a reliable and efficient backend infrastructure was a crucial step. Let's explore some code snippets that demonstrate key aspects of the backend implementation:

Route Configuration:

typescript
const sirasa = require("./routes/sirasa.route");
const hiru = require("./routes/hiru.route");
const itn = require("./routes/itn.route");
const mirror = require("./routes/mirror.route");
const derana = require("./routes/derana.route");
 
app.use("/sirasa", sirasa);
app.use("/hiru", hiru);
app.use("/itn", itn);
app.use("/mirror", derana);
app.use("/derana", derana);

By separating routes into their respective modules, we ensure a clean and organized structure. Each route module handles the specific scraping and retrieval logic for the corresponding media website, making it easy to add or modify routes in the future.

Caching Mechanism:

typescript
if (articlesExistInCache) {
  // Fetch articles from MongoDB cache
  const articles = await Article.find({ route: currentRoute }).exec();
  res.json(articles);
} else {
  // Scrape articles from website using Cheerio
  const scrapedArticles = scrapeArticlesFromWebsite();
 
  // Store scraped articles in MongoDB cache
  await Article.insertMany(scrapedArticles);
 
  res.json(scrapedArticles);
}

This code snippet demonstrates the caching mechanism implemented in Sided.News. When a user visits a specific route, the backend checks if the articles for that route exist in the cache. If they do, the articles are fetched directly from MongoDB. Otherwise, the backend scrapes the articles from the corresponding website using Cheerio, stores them in the MongoDB cache, and returns the scraped articles as the response.

These snippets provide a glimpse into the inner workings of the backend infrastructure of Sided.News. Stay tuned for more insights as we dive deeper into the frontend implementation and additional features that make this application a powerful tool for comparing news articles from various sources.

Challenges

Building the Sided.News application came with its fair share of challenges. Let's explore some of the key hurdles encountered during the development process:

  • Data Consistency and Reliability: Ensuring the consistency and reliability of the scraped news articles from different media websites posed a significant challenge. Media websites often have varying HTML structures and data formats, requiring careful handling and parsing techniques to extract accurate and reliable information.
  • Handling Dynamic Website Content: Some media websites use dynamic content loading techniques, such as AJAX or JavaScript rendering, which made it more challenging to scrape the desired data. Implementing mechanisms to handle dynamic content loading and retrieve the complete set of news articles required additional effort and careful consideration.
  • Scalability and Performance: As the number of users and news sources increases, maintaining a high level of performance becomes critical. The application needed to handle a growing user base and efficiently manage the scraping and caching of articles from multiple media websites. Optimizing the backend code, implementing caching mechanisms, and leveraging efficient database operations were key factors in achieving scalability and optimal performance.
  • User Experience and Interface Design: Designing an intuitive and user-friendly interface that allows users to navigate between different media sources and compare news articles effectively required careful consideration of user experience (UX) principles. Striving for simplicity, clarity, and ease of use were ongoing challenges throughout the development process.

Future Work

Looking ahead, there are exciting possibilities for future enhancements and expansions to the sided.news application. Here are some potential areas for future work:

  • Bias Analysis: Adding a bias analysis algorithms and methodologies can provide users with more accurate and insightful evaluations of news articles. Implementing advanced natural language processing (NLP) techniques and machine learning models can help identify nuanced biases and provide a comprehensive analysis of media reporting.
  • User Customization and Preferences: Empowering users to customize their news experience based on their preferences and interests can greatly enhance engagement. Implementing features such as personalized news feeds, topic filtering, and bookmarking functionality can allow users to tailor the app to their specific needs, making it a more personalized and valuable resource.
  • Social Interaction and Community Features: Introducing social interaction elements can foster a sense of community among users. Implementing features like comment sections, user ratings, and sharing capabilities can encourage discussions, promote information exchange, and further enrich the user experience.
  • Real-time Updates and Notifications: Incorporating real-time updates and push notifications can keep users informed about the latest news developments. By integrating technologies such as WebSockets or server-sent events, users can receive instant updates on breaking news, ensuring they stay up-to-date with the most recent information.
  • Expanded News Sources: Expanding the range of news sources available on Sided.News can provide users with a more comprehensive view of the media landscape. Continuously adding reputable and diverse media outlets, both nationally and internationally, can enhance the platform's credibility and offer users a broader perspective on news events.
  • Data Visualization and Analytics: Implementing data visualization tools and analytics features can help users interpret and understand news trends and patterns. Presenting data-driven insights, such as visualizations of bias distribution or sentiment analysis, can enable users to make more informed assessments of media reporting.
  • Mobile Application Development (already done): Building a dedicated mobile application for Sided.News can extend its reach and accessibility, allowing users to access the platform on their smartphones or tablets. A mobile app can provide a seamless and optimized experience, leveraging native device capabilities and enhancing user engagement.

References

Enjoyed this post? Read the next one ↗