New AI Model Maps the Evolution of Political Memes

In 1982, a simple smiley face— ‘😊’—appeared on an online bulletin board at Carnegie Mellon University to flag humour. Combining text and a symbol, this three-character emoticon became one of the first examples of Internet memes.

Four decades later, memes have evolved beyond simple comedic expressions into strategic tools of political communication. For example, in 2017, the #MeToo movement gained global momentum, becoming one of the most powerful social phenomena born online. As the movement spread, it led to a wave of political and cultural memes that amplified, critiqued, and influenced public discourse around the movement.

However, despite their popularity, the detection and evolution of memes remain significantly understudied. How do we capture something designed to mutate?

To answer this, we created ‘Meme-Hunter’—a multi-modal deep learning system designed to detect, classify, and trace the life cycles of political memes online. This study specifically applies Meme-Hunter to the 2018 US midterm and Swedish national elections to understand how these visual-textual fragments spread across cultures and subcultures.

The origins of memes 

The term ‘meme’ predates the Internet. It was coined by evolutionary biologist Richard Dawkins in the book ‘The Selfish Gene’ (1976) to describe cultural units that spread through processes akin to genetic evolution—variation, selection, and replication. Our research extends this foundational concept, validating the evolutionary characteristics of memes.

The proliferation of memes, as we know them today, began in the early 2000s. They quickly became a distinct form of online expression. Social media accelerated their reach. Memes became faster in mutation, more visual, and increasingly anonymous. Unlike traditional creative works, they rarely carry attribution, allowing controversial content to travel freely and without consequence. The blend of anonymity and adaptability makes them especially effective for political messaging and propaganda.

Building Meme-Hunter 

Meme-Hunter was created to find images online and classify them as memes vs non-memes by integrating computer vision, optical character recognition (OCR), and facial recognition. To begin classifying images, the system focuses on two common meme formats: 

1. Pictures with white text in the Impact font 
2. Images with text embedded in a white box overlay 

These visual cues, the extracted text and facial data, help determine whether an image qualifies as a meme. Since meme text is often stylised and high-contrast, the team developed a custom image-cleaning method. For example, if a meme has white text on a dark background, the image is converted to black and white and then inverted. This helps the OCR tool Tesseract to extract the text more accurately.

To complement text extraction, Meme-Hunter uses facial recognition using an open-source face detection library. Since political memes frequently feature public figures, facial encodings improve the classification process, particularly for identifying politically charged content. The team tested four classification models:

A text-only classifier using a Long Short-Term Memory (LSTM) network 

1. An image-only classifier using Convolutional Neural Networks (CNNs) 
2. A multi-modal classifier combining text and image 
3. A multi-modal classifier integrating text, image, and facial features 

In the final model, the outputs of the LSTM (for text), CNN (for visual features), and facial encodings are combined into a single vector and passed through a dense neural layer. The joint deep neural network (DNN) model consistently outperformed the others.

We built a large and balanced dataset of over 50,000 images (25,109 memes and 25,100 non-memes) scraped from Reddit, Twitter, Tumblr, and Instagram to train the model. We ensured that even non-meme images included text and photos to prevent the model from conflating text presence with ‘meme-ness’.

Our approach works even when memes change format or style, unlike older systems that could only spot memes by matching them to known templates. Meme-Hunter detected nearly eight times more memes in election datasets, making it particularly effective for studying cultural change at scale.