Perched on the edge of a cliff on a tropical coastline, a villa with tall glass windows and a large infinity pool takes your breath away. It looks like a fantastic holiday destination, or at least a perfect movie set. That towering limestone rock jutting out of the ocean does ring a bell. Isn’t that where James Bond had a pistol duel with “The Man with the Golden Gun”? Thailand, right? Well, yes and no. With artificial intelligence generating regurgitated concepts, everything seems to be real. And yet – it isn’t. In the era of social media scams, deceiving Facebook pictures like this one have become helpful tools for detecting AI fools.
Bora Bora’s box of AI deceits
It’s easy to buy into the idea of a luxurious retreat surrounded by lush vegetation. There’s even some blurred basil in the foreground so that you can feel “at home”, just about to cook a recipe for a perfect vacation, a blend of exotic and familiar.
While you could imagine Roger Moore sipping cocktails here after shooting his movie scenes in 1974, this is not Koh Tapu in Phang Nga Bay, Thailand.
The coastal villa is in – Bora Bora, more than 12 thousand kilometers away, oceans apart. At least that’s what the Facebook page Slovak Travel claimed when they shared this image, describing it as “Bora Bora Paradise🩵”. The post raked 10 thousand positive reactions, mostly thumbs up, hearts, and wow emojis.
“Awesome”, “Gorgeous”, “Amazing”, “How much?”… More than a thousand oohs and aahs poured in, with admirers expressing readiness to move, or at least celebrate their special anniversary there.
Check out the James Bond scene in which the original Koh Tapu was not modified by AI!
Facebook newsfeed is swamped with AI images pretending to be photographs, delivering the world populated with avatars, Pandora’s box of deceits
“This is not Bora Bora”, said those who visited French Polynesia. “This is James Bond Island”, said the experts on Southeast Asia. But the depicted island never starred in any movie. The dream vacation place never existed. It wasn’t even a photograph, but an AI creation in Midjourney, prompted by Maria Dudkina, an online personality known as @sunt_mrr, and then misrepresented as Bora Bora by Slovak Travel.
These days, Facebook newsfeed is swamped with similar AI images pretending to be photographs, delivering the world populated with avatars, Pandora’s box of deceits. It’s not just entertainment, but a sneaky way to monetize people’s naivety. Before image generators appeared, we had to raise our skeptic shields mainly on April Fools’ Day. Now, staying suspicious is an everyday challenge. How to not end up as AI fools?
From London Eye to Scotland AI
Fraudulent minds quickly embraced opportunities provided by AI, like low-hanging fruit. What required at least some photoshopping and copywriting effort a few years ago, was now bringing easy money with a click of a mouse.
One of the most illustrative examples of this practice was the recent case of Willy’s Chocolate Experience in Glasgow. Billy Coull from the House of Illuminati used AI visuals to promote a Wonka-inspired wonderland but served up a wonky disaster that infuriated hundreds of families. The event immortalized characters of a meth-lab Oompa Loompa (played by Kirsty Paterson), and an evil chocolate maker The Unknown (Felicia Dawkins).
In a weird boomerang effect, Facebook responded by dosing the same medicine. Alan Livie shared images of statues commemorating the Bored Meth Oompa Loompa and The Unknown, supposedly erected in Glasgow’s city center just a week after the event. These AI creations were obviously satirizing the monumental disaster. They received 12 thousand likes and over a thousand comments, mostly from people who understood the joke. Still, in an aha moment, some users felt the need to state the obvious: “More AI art people. Learn to recognize it.”
Livie’s posts were published with a different intention than that fake Bora-Bora pic. They were a sarcastic response to reality, shared in Travel Scotland Goals, a Facebook group known for welcoming clearly fake or misinterpreted photos and self-deprecating jokes about Scotland. You can find anything from Scotrail‘s time-saving Northern Line on the mountain cliffs of Fifeshire (beat that, Hogwarts Express!) to a gigantic spa hidden inside Edinburgh Castle.
Even when images are not executed perfectly, you’ll still notice intrigued foreigners planning to visit the medieval wellness facility “next time”, or at least asking “does this really exist”. But locals will also have brilliant answers, such as: “Looks like another fake photo, to me. I know for a fact that’s Whinfell Forest Center Parcs because my son used to be a lifeguard there.”
On March 10, this group was the birthplace of the now-cult post about relocating the London Eye to Loch Lomond in Scotland, after the 25-year-long lease expired. It used real photos from the Eye’s construction in 1999, but the “news” created by Paul Gilchrist sounded so convincing that it infiltrated serious circles and was debunked by numerous fact-checkers (from Snopes and Mashable to Reuters and Facebook) before being taken down. It resurfaced again like a virally shared April Fools’ Day joke.
Some of my AI images went viral on Facebook recently, gaining thousands of likes and comments. And all thanks to - Eurovision.
What’s with all the AI photos on Facebook?
Travel Scotland Goals may be a benevolent oasis of humor that Facebook’s confused algorithms sometimes propel into mainstream feeds, out of context. But there are Facebook groups, pages, and profiles that employ AI without having entertainment in mind.
So far, the risks associated with the use of AI image generators have mainly been discussed over the issues of dehumanizing celebrities (Taylor Swift’s deepfake porn), or humanizing dangerous political interests (the black community seemingly endorsing Donald Trump or AI Joe Biden discouraging people from voting).
But this March, Renee DiResta (Stanford University) and Josh A. Goldstein (Georgetown University) researched how profit and clout-motivated page owners use unlabeled AI-generated images on Facebook for spam and scams. They found not only that these posts gain unusually high volumes of engagement, but that many Facebook users don’t seem to be aware of the synthetic origin of such images.
The cybersecurity data from Sumsub‘s report shows us that AI-powered fraud is on the rise, with detected deepfakes increasing by a whopping 10 times from 2022 to 2023. The highest surge is seen in North America (1740%), with Canada delivering the largest contribution to this percentage (3000% increase). The most affected industry is online media, where the identity fraud rate had a 274% rise between 2021 and 2023.
As the largest social media platform, with over 3 billion monthly active users (and 2 billion logging in every day), Facebook became a natural breeding ground for AI scam & spam.
Today, it is almost impossible to open the Facebook app and not find the endless scroll of AI-generated images. Tiny houses by the sea, log cabins by waterfalls, fluffy unicorn bedrooms and cheese-themed bathrooms, sympathy-calling children’s handicrafts and grandma’s crochet creations, impossible food designs, non-existing wildlife, adorable babies dressed in cabbage, and why not – the most exotic tourist destinations.
With optical illusions that make you squint to see Jesus or Hitler, and even those Fibonacci locales where houses masterfully wind in a continuous curl, the AI flood is literally spiraling out of control. But where does it all lead?
Pixels gone wild in Santorini
It is one thing to just laugh at the Pope fashionista strutting down the streets of Rome in a Balenciaga puffer jacket. But beyond a Midjourney exercise of a construction worker tripping on magic mushrooms, a more sinister spam/scam scene lurks in the shadows. Forget the days of random emails from Nigerian princes; scammers have found an easier way to detect their next prey, people who would fall for anything.
The first time I noticed the AI Facebook trend with fabricated travel content taking off was in September 2023. Just a year after modern text-to-image diffusion models appeared on the scene, they were entering the mainstream but were far from refined.
The not-really sophisticated image of “vacation in Santorini“, with poorly mimicked Greek architecture connected by waterslides, was clearly made in the infancy of the image generator’s development. The architecture didn’t make sense, stairs led to nowhere, people were just skin-toned blobs… Yet, shared on the Facebook page A World of Luxury Travel, the post attracted nearly a million likes, more than 58 thousand comments, and over 84 thousand shares.
“Maravilhoso”, exclaimed Ana Paula from Portugal. “So schön”, added Heiner in German. “Tellement beau!!! 😊👍”, jumped in Annie in French. “我要去”, proclaimed Chang. One could practically advance in languages while dreaming of visiting the fantastic waterpark on the Greek island.
People were adding the place to their bucket lists, planning girl trips and honeymoons. “Maybe after Tahiti”, mused practical Michelle.
Those who had just returned from Santorini, such as my fellow Croat Mario, knew what they had to do. “We missed this🤔 Let’s go again😜😘”, he said while tagging his friend. “Oh, yes! 🥰”, Ana responded.
Even reading someone saying “This isn’t real” seemed surreal. Anyone interested in Oia, the home of magical Santorini sunsets, knew this AI nonsense had nothing to do with Greece. But the already mildly critical eye should have been able to detect that everything was nonsensical altogether.
If you want to see the Pope devouring chocolate and Vladimir Putin riding a bear, check out Pipeaway's world of chocolate, clearly labeled as AI!
From clickbait to cash grab
The non-existent Santorini is an interesting example because the fantasy island revealed itself as fertile soil for both spam and scams.
Firstly, it showed how AI pictures on Facebook can replace the earlier tactic of clickbait titles. After social media degraded the visibility of posted links, AI-generated images, showing attention-grabbing potential, started popping up in newsfeeds like mushrooms after a rainstorm. The page admins, interested in profiting from websites overpopulated with ads, would then promote their off-platform content in the comments of the post gone viral.
The model is very similar to the one exploited by the Macedonian fake news industry in 2016. Veles youngsters used Facebook groups as a place to recruit visitors to sensationalist pro-Trump websites that heavily monetized this traffic.
In the case of A World of Luxury Travel Facebook page, their links would lead to mstfootball.com, and other shady domains, with low-quality AI-generated content and high-yielding advertising space.
Millions of people curious to learn more had an opportunity to click on these sites, see even more AI images of Santorini, and allow Google to reward the spammers with ad money.
From likes to lies
On the other hand, the Facebook home of AI Santorini revealed that it can serve as a litmus paper for identifying potential scam victims too.
When she saw Greece of her dreams, a divorced woman from Portugal named Laura commented: “Wonderful.” Robert Micheal, conveniently a widowed Ukrainian, swooped in: “Hello I’m just new to Facebook and I really like your profile can you please send me a request so that we can get to know more about ourselves 🌺🌺🌺🌸🌺”
Malinda, a single mom from Oklahoma, couldn’t contain her excitement at the sight of Santorini’s waterpark: “It’s absolutely magical! 😍” Harold Griffin – a divorced man from Texas, of course – catfished back: “Apologies.. for dropping this here but it feels too great to visit your timeline, I always enjoy what you share here on Facebook but we’re not friends yet, I have tried several times to send the friend request but it’s not going through. Do you mind trying from your side? I will be happy to be friends with you. If you find this message embarrassing please pardon my manners Thank you…”
When Niina from Finland (who had wedding photos in her album) asked “In which village is this?”, she didn’t get a version of Tinder Swindler in her mailbox. Instead, certain Clement Gabriel approached her interest in an invented Santorini by promoting his manager, a supposed expert in bitcoin mining and crypto trading: “I saw a post about his company and l decided to give a try of my 3000 pesos to my greatest surprised l make withdraw of 30,000 pesos to my gcash account he is very reliable and free to communicate with, just click on the link below and Contact her 👇👇👇👇👇👇👇🏼👇🏼👇🏼👇🏼👇🏼”.
We also had Ramona from Romania who used the AI Facebook post on Santorini waterpark to ask for help for her 23-year-old brother Mihai, supposedly looking for work: “Because there is not very good internet in the country, he asked me to leave his number here. (…) Please, if you need a hard-working boy, call his number.”
Scammers hardly needed to lift a finger in their search for targets; gullible people were practically lining up under the AI image, like moths to a flame. What you read here is just an introduction to asking for personal details and sometimes selling products that do not exist.
There are plenty more catfish in the sea
Why do people fall for this stuff?
Just like in the case of lost luggage scams, these Facebook pages target weak spots in human brains by presenting social proof.
Pages studied by DiResta and Goldstein had a mean follower count of 128,877. The images that came under their scrutiny received hundreds of millions of engagements and exposures. One of these AI-generated posts racked up a mind-boggling 40 million views and over 1.9 million interactions, entering the top 20 of the most viewed Facebook content in Q3 2023.
Facebook reported that, only in the fourth quarter of 2022, it removed 1.5 billion fake accounts. But among the sea of commenting bots, we can also find real people praising the AI content
Nowadays, people mostly access social networks on their mobile phones. Small screens and a general tendency to tone down or switch off critical thinking in chill mode don’t provide the best conditions for zooming into pictures or doing thorough research on something that seems benign.
There are also dozens, hundreds, and sometimes even thousands of people admiring/approving the content. In certain niches, at least my research of the culinary AI images suggests this, it’s even hard to find a person expressing doubt in the artificial content – most users uncritically drool over these fake desserts, and many will follow links to websites spitting out doubtfully doable recipes quicker than you can say ‘fast food’.
Many of the praises are left by AI bots. Some are easily detectable, such as this brainless comment under the waterslide Santorini: “Very Good !!! Very Interesting !!! Very Amazing !!! Very Awesome !!! There’s no other words that I can say other than that, all the stories, acting, drama, and actors are absolutely perfect. I love this movie and really really enjoy it.. Thanks for this movie.”
Facebook reported that, only in the fourth quarter of 2022, it removed 1.5 billion fake accounts from the network. But among the sea of bots pretending to be humans, we can also find real people. Powered by bragging rights, they are supporting a clear AI hallucination. Just read some of the comments under a fake Gaudi house that gathered 88k likes on the Facebook page I Love Spain:
“Saw this myself in Spain and it is just spectacular.”
“I saw this, there were a few houses like this. Very cool.”
“I saw this building in person in Spain and just loved it. I was there during the week that 9/11 and had to see how I could get out. My cousin sent a fax to my hotel and opened her home to me in England.”
“It is a beautiful place, I visit soo many times. Never enough.”
“Been there, the most awe struck I’ve ever been, a true trippy place. I LOVE Barcelona!! AND GAUDY!!!! 😍❤❤❤❤”
“Been there it is amazing!”
It’s one thing to fall on all those “fantastic”, “gorgeous”, “sublime” and “it definitely stands out” comments. But when there are real people claiming to have actually been at the site, well – it must exist!
Sharon, an older lady who hadn’t had the chance to visit the Disney-like nightmare of Gaudi yet, could only express her admiration: “I love it. How did they ever get someone to build it?”
The widowed orthopedic surgeon at the UN, Mark Donald, had a ready-made answer for exactly her type: “Hi! I admire your value for different perspectives, can we be pals? Add me up if interested.”
Artificial intelligence can be used for better things than deception. For instance, it can help us imagine the original 7 wonders of the world!
Slovak Travel for your dream fauxcation
Let’s go back to where we started – the Slovak Travel Facebook page, a digital creator behind the Bora Bora-Thailand crossover viral post.
Since January, they’ve been bombarding our feeds with images of exotic-looking destinations, with Maldives, Seychelles, and Bora Bora gaining the best traction. The vacation buffet always came accompanied by colorful hearts, and brief descriptions such as “paradise” or “the vacation we all dream of”.
For a page that bears a European country in its name, it certainly prioritizes tropical getaways, from Aruba to Zanzibar. But Slovakia is one of the AI-fraud champions on the Old Continent. From 2022 to 2023, the number of deepfakes has grown 1,500%. That’s almost double the European average, and puts Slovakia among the top countries exposed to AI frauds, just behind Belgium and Romania.
Slovak Travel’s digital footprint tells us that the page was launched in 2022, but then it was named Epic Travel, just like one private aviation company.
They started posting the majority of the content visible on the page in January 2024, when the page rebranded as Tiptravel, practically leeching off TIP Travel, a legit Slovakian travel agency specializing in Mediterranean and exotic tours.
In March 2024, the administrator changed the page name again – to Slovak Travel. The new name is eerily similar to the official tourism agency of Slovakia. In fact, from Slovakia Travel, it differs only in two letters – ‘ia’. Or AI.
With the help of technology, Slovak Travel amassed more followers in three months than Slovakia Travel managed to do in 13 years.
Sunny side down – cracking the Slovak scam
After teasing us with images of exotic turquoise waters for several months, Slovak Travel Facebook page finally revealed where it’s heading.
Last week, they published three giveaway posts, advertising a spa retreat in Turčianske Teplice, a trip to Turkey, and of course, a holiday in the dreamy Maldives. All you had to do to claim your prize was share the post to other Facebook groups (over a thousand people complied), and write ‘done’ or a number of eggs (1 to 4). Don’t ask why.
When Beata chose to answer “1🥚”, handsome Christian Skari appeared to serenade her: “Hello beautiful, did you sleep well, have a nice day and you are very beautiful, I want us to be friends.”
Certain Carole Hamousin saw an opportunity to slide in with the following message: “👋 Are you looking for financial support? Take advantage of our flexible loan offers at competitive rates. Get quick and easy funding to make your projects a reality. Contact us for a tailored solution to your credit needs. 😇❤️”
But beyond just selling tinder-swindler kind of love to middle-aged women and showering financially struggling individuals with loans, Jan A. revealed the scam scheme: scammers approach “lucky winners” on Messenger and then try to clear their bank accounts.
Dana R. confirmed this: “Who could believe this? They will tell you that you have won and they ask for complete account and card details to get to your euros, cunning.”
UPDATE: As of today, the mentioned giveaways on Slovak Travel were deleted, and the new ones appeared, offering a family holiday in Mallorca, and a "4000 thousand voucher", whatever that is.
AI and Facebook – a love story gone haywire
The issue with AI and Facebook photos is that they seem to be in a monstrous marriage – a smiling princess feeding an insatiable beast. Or vice versa? Once this content enters your system, it replicates faster than a virus.
During the period of my research in the last months, I noticed that the more AI-generated images on Facebook I saved, the more they appeared in my feed. This aligns with Stanford-Georgetown researchers’s experience. They suspect the explanation could be in Facebook’s algorithm promoting content that is likely to generate engagement, even when it’s the content from the pages users don’t follow. In fact, while the visibility of pages that users follow plummeted down since 2021, unconnected posts rose from 8% of feed views in 2021 Q2 to 24% in 2023 Q3.
Last week, Meta announced the plan to roll out a “less punitive” policy for AI-generated content. Reading such a title made me chuckle, as I haven’t noticed many penalties in that field so far. Apparently, the idea is that instead of removing the problematic content (and risking “unnecessarily restricting freedom of speech”), Meta will slap a label on AI images “that it could detect”. To do that, they will rely on industry-shared signals and people self-disclosing such content (as if the lack of that self-disclosing wasn’t the essence of the problem).
Facebook AI scams are lucrative businesses, often even purchasing ads from Meta
Judging by the content it serves my feed nowadays, I can conclude that Facebook is definitely able to detect generative AI, as it continues to serve me such content from pages I didn’t even know existed.
The AI fraudsters found a cozy home on Facebook. Just like fake lost luggage pages, these scams are lucrative businesses, often even purchasing ads from Meta. Even if the company promises to “reject an ad if it contains debunked content”, they still require disclosing the digital manipulation only when it’s a political or a social issue ad. So unless we’re being served ads about elections or some earth-shattering social agenda, it doesn’t seem anything will change with all those non-existing paradises, crocheting grandmas, and crab Jesus.
Falling for AI Facebook pictures – Conclusion
With a shift of generations, Facebook may start to resemble an online retirement home. However tempting, it would be oversimplified to point fingers at baby boomers (born 1946-1964) and generation X (1965-1980) as easily fooled victims.
Not only aging users are in jeopardy from fraudsters. Earlier studies have shown that when it comes to spotting fake news, young and older adults are quite comparable. Media literacy and critical thinking should be a priority for new generations too.
Under the AI image of seashell-shaped cottages by the sea, young and old alike gush over its idyllic allure, calling it a dream destination in the comments. They’re blissfully unaware of how correctly they describe it.
The same dreamlike AI Facebook picture shared on the Maldives Islands page gained 262 thousands likes from equally impressed visitors. Natalie wrote: “Wow, absolutely beautiful. I can hear the waves and smell the fresh ocean breeze.”
But that sound and that scent – it’s all in our heads. While we were worrying about our prompts making large language models hallucinate, we failed to notice that AI made us hallucinate too.
Behind the cloak of seemingly harmless entertainment, AI Facebook content became a sneaky prelude for stealing personal details and passwords. The spam/scam industry transitioned from clickbait titles to African kids crafting the Son of God from vegetables (“made it with my own hands”) and dreamy holidays in Maldives or Bora Bora that will never become a reality.
Beware, AI fits perfectly into naivety. Just behind ‘n’.
What do you think about the flood of AI Facebook pictures?
Share your thoughts in the comments, and pin this article for later!
The Santorini slides look quite appealing. That said, it is scary how easy it is to create AI based “places” that can be used to scam people.
Thanks for your comment, Sonia!
The technology is advancing rapidly, and our confusions will only grow.
Just yesterday, I was listening to a new AI-generated music tool, with lyrics and all. And – wow!
We are getting more and more easily convinced that what we are seeing and hearing is true nature of things.
Lines between reality and fantasy are getting blurred, and this is the perfect place for those who prefer hunting in the dark.
I guess we just have to learn to live with it, and train our defence mechanisms for new challenges.
The Internet has been awash with misinformation all along, but with the advent of AI, it’s become even worse. I’ve ex;perimented with it to search for background information on blog posts, and sometimes it works well. But I’ve soon realized that everything needs to be fact-checked. Somewhat ironically, as I was seeking background information on something a few days ago, the information AI returned was from one of my very own blog posts. Regarding the photos you’ve shared, I would like to think that people would realize they were fake, but I’m not so sure. There are some pretty naive people out there!
The internet has indeed become a breeding ground for misinformation. Even fact-checking information online requires caution, as we may find sites that confirm the fabrications, simply because their content was also created or co-created by the AI.
All of us need to keep up with the technology, but we also need to upgrade our critical thinking in this digital age.
As for the AI-generated photos shared here, it’s concerning how easily they can deceive people, especially those who may be more trusting. So indeed, it even seems that one doesn’t need to invest a lot of effort to achieve a successful scam career nowadays. Sadly.
Thanks for sharing your insights, Michael!