LLMs—especially Nano Banana Pro—are really, really, really bad news for online radicalization. The primary function of Nano Banana Pro is to take text prompts and use the words to create images, diagrams, infographics, and character caricatures. It is the successor to the “Nano Banana” (Gemini 2.5 Flash Image) model. Why does it have such a stupid name? It was a placeholder and when everyone loved the beta version, they kept the name.
Unlike older image models that simply matched pixels to keywords, Nano Banana Pro uses the Gemini 3 Pro LLM as a “reasoning core.” If you ask for “a diagram of a frog’s life cycle,” it doesn’t just hallucinate random shapes. It uses the LLM to research the actual stages (egg -> tadpole -> frog), plans the layout, and then generates the image. It also uses tools like Google maps and Google search results to inform its image generation.
I uploaded a photo of myself and a photo of some teens that everyone involved in school safety should recognize. My prompt was “make a four panel cartoon of me and my friends having fun at school”.
The bottom right panel that looks just like the icon Columbine High CCTV footage from the attack is beyond words. There are absolutely no guardrails on these AI programs that are available to anyone.
The mechanics behind how Nano Banana Pro created this very disturbing image explains the foundational flaws in using probabilistic LLMs to produce “truth” or “facts”.
When I asked Gemini to “add me to this picture of my friends”, the image generator put me into the center of the cartoon school shooter line-up...with my correct height listed. But I didn’t give Nano Banana this info.
My prompt: How did you know for the photo that I’m 6’1”?
Gemini: That was a complete lucky guess by the image generation model!
Gemini: I do not have access to your personal stats or height. However, the reference image you provided (the cartoon group) had a very specific style where every character was labeled with a height. When the AI added you to the lineup, it tried to match that pattern. It likely placed your character between the “6’3” and “5’9” figures to balance the composition and then assigned a random number (”6’1”) that made sense for that visual scale. It’s a pretty wild coincidence that it got it exactly right.
The “logic” that led to the model guessing my height correctly is Probabilistic Pattern Matching. At their most basic level, these models are not “thinking” or “knowing” facts about the world. They are prediction engines trained to complete a pattern based on what is most statistically likely to come next.
This logic also explains why AI hallucinates. If you ask: “What is the capital of Mars?” The pattern of the sentence looks like a geography question and geography answers usually look like two proper nouns. A LLM might answer “Mars City” or “Olympus Mons” not because it’s true, but because those words fit the linguistic pattern of an answer, just like 6’1 fit the visual pattern of the cartoon.
From the prompt photo, Nano Banana saw a pattern where everyone has a height label, so it picked a number that matched a statistical probability, not a retrieved fact. While it’s easy to be amazed by how much AI “knows”, we need to realize that every “correct” AI response is luck and probability instead of objective truth.
The implications of Nano Banana Pro go far beyond a lucky guess about my height because they signal a terrifying new frontier for online radicalization. By seamlessly integrating me into a lineup of the Columbine shooters—complete with a chilling CCTV perspective—the model demonstrated that its advanced “reasoning core” lacks the most basic ethical guardrails. It didn’t recognize the iconography of mass violence, it simply recognized a visual pattern and completed it.
This is the fundamental danger we face when we are mistaking a machine’s statistical proficiency for objective truth and moral understanding. When an AI operates purely on “autocompleting” reality based on probability, it validates whatever prompt is fed to it, no matter how heinous. If these tools cannot distinguish between a “fun school cartoon” and the glorification of tragedy, they are not just creative engines, they are accelerators for violent fantasies and available to anyone with an internet connection.
If you missed it last week, this is how easy it is to trick ChatGPT into plotting a school shooting: https://k12ssdb.substack.com/p/chatgpt-helped-me-plan-a-school-shooting
David Riedman, PhD is the creator of the K-12 School Shooting Database, Chief Data Officer at a global risk management firm, and a tenure-track professor. Listen to my weekly podcast—Back to School Shootings—or my recent interviews on Freakonomics Radio and the New England Journal of Medicine.