The Machine Says I Wrote This. You Still Have To Decide If It's Any Good.

Flock cameras proved with 100% accuracy that Chrisanna Elser stole a package from a porch.

Flock got it 100% wrong.

An officer accused Chrisanna of theft. He had the video to prove it. When she asked to see that proof, he told her no.

He stood at her door and accused her and told her she’d have to go to court to see the evidence. Guilty until proven innocent.

Over the next two weeks, she collected and provided evidence that proved her innocence. Case closed.

The case raised Fourth Amendment questions, due process issues, and ire at the smug assurance displayed by the officer. He was so certain that Chrisanna was guilty.

Pangram AI detector on Substack is much like Flock. For writers.

This is very much a Substack issue. (For those who don’t know, Substack is a publishing platform for writers and creators.)

Substack attached and released Pangram as a feature for users to determine the percentage of writing that was human, AI-generated, or AI-assisted.

The framing was trust. So the reader can determine that what they’re reading is human ideas and writing or AI-generated.

Pangram As Theater

There are brilliant articles that cover what happened and what you can do to have that feature disabled at your account level. I suggest that you read these if you want to dive deeper into the mechanics of it all.

In all of the furor, I asked myself “What’s the big deal? Who cares if people scan for AI?”

I have been reading online from the days before AOL and Compuserve, when digital bulletin boards were a thing. When I read now, I occasionally copy/paste the content into AI to test for “humanness.” Usually to confirm what the writing showed me.

To assure myself that what I’m reading is authentic and not some hallucinatory slop cobbled together by AI. To protect myself from lies and bad information.

Which is simply theater for my own entertainment.

If I use AI to determine the AI to Human ratio of writing in a piece, then what? Cluck my tongue and drone “Naughty naughty” or silently judge them? Or reject their writing because it’s not “real” enough?

If a creator uses AI and presents ideas as true, and they are, does the mechanism change the truth of the idea? Or if a writer crafts a lie or half-truth in writing that sells the idea as true, and I buy it, does the mechanism change the lie?

So what if AI wrote the words? If the idea resonates in me, stirs me to action, inspires me to become better, then is my transformation lessened because Claude or Chat-GPT helped the creator?

Sure, this raises other ethical questions from the use of Large Language Models (LLM).

Like, whose idea is it that is moving me? The creator using AI, or the LLM? Part of the tension arises from creators pulling from AI and passing the ideas off as their own.

Provenance of the truth matters.

It’s how we define plagiarism. Stealing my work and passing it off as your own is stealing. If AI is drawing from its repository of writing, is that plagiarism? Is it stealing if we cannot distinguish my writing from ideas passed from an LLM, from Montaigne’s writing, or Baldwin’s, or anyone else’s specifically without quoting them?

There is nothing new under the sun.

You know this. I know this. We all know this. I’ve written this here as had Solomon some 2500 years ago. This idea, this truth, and its expression is not new. Yet, in the context of this article, this string of ideas I’ve put together, this “not new statement” is new. Ish. New enough.

Would Pangram flag this as 100% human?

What’s The Percentage Mean?

I tested my last article with Pangram. Pangram flagged it as “fully human-written.” I got 100%! Wipperdoo.

What does that even mean?

I can attest that I wrote every word in my article. But with something to sell (my ideas) my attestation is suspect, isn’t it? So now you have a third-party to confirm my claim.

But I didn’t make a claim. I simply wrote and posted.

Nowhere did I include “100% written by me.” I could write “30% hand-crafted with genuine gel ink on acid-free paper.” Or “100% touch-typed for computer use on a genuine Mac computer.”

These would be true claims. And none of them make a damn bit of difference. Not to me and certainly not to you.

My process, whether hand crafted or pissed in the dirt, is my process. What I present is for you to read. Or ignore. To consider my ideas and accept or reject them.

What you do with the writing presented is on you. As it should be.

Sure, Pangram is presented as a tool to help you in the process. To pre-screen against shenanigans by giving you a percentage. A number. A score of who maybe, probably, likely, here’s a damn good guess based on our algorithm and computery stuff that lets you know a person wrote this.

Cool. (awkward silence)

What does that mean? Does that build trust for you? How?

What prevents a person from taking AI output, typing those words into their word processor, maybe massaging a phrase or word here and there, copying it to bypass Pangram, skirting the watermarking in Claude, and posting it as their own? Would this be AI-assisted or AI?

A person determined to bypass detection will work to bypass detection. But I think this is a minority user. I know brilliant creators who post useful, imaginative, creative ideas. And they use AI in their workflows. Some even have AI write their articles fully. And I enjoy their writing. AI writing.

AI has its issues. But letting people invoke AI to determine if AI wrote an article doesn’t make much sense. If AI is a problem, shouldn’t the reader determine that for themselves?

The Burden Was And Remains On The Reader

My favoritist teacher of all time was one Sister Michele, a Sister of the Holy Faith. She taught me through seventh year. She was also my first real crush, but that’s a story for another day.

She pushed me to read as if my life depended on it. Her brogue rendered my name Danny as Denny. “Denny, you’re a thinker. Reading helps you think better.”

She introduced me to thinking critically. I remember reading articles from Newsweek, The Atlantic, and Time. I’d break down the thesis, show the supporting ideas, and outline the flow of the article. It was grueling. And fun.

I loved seeing the ideas pop out of the text. It opened vistas of thought and ideas and places and people. And she used the process to teach me how to see the ideas without drinking the Kool-Aid. To see the writing as the writing.

As a writer, I have felt a profound burden to bring truth. Fiction or non-fiction, I work to ensure that people experience a sense of what is real. To stand up to scrutiny.

I get fired up when my writing stirs people to action. Or connects them with their emotional reality in a way that reminds them of their humanity. KD did this to me writing about a song. A SONG!

But there are writers who work to populate a content calendar. Or drive SEO. Or fill a feed on their website or Substack site. Good enough is good enough. And for a large number of readers, good enough is…well…good enough.

People love to read listicles. And love the dopamine hit of “How to” stories. Even if it’s the same “how tos” rehashed, recycled, and recast. If it catches my eye and I read it, then there’s something there I want.

And it’s on me, as the reader, to determine if the writer held up his end of the bargain. Did he deliver on his promise? Or do I quit reading because I recognize I’ve seen this useless crap before?

I’m not sure how scoring an article with Pangram helps me determine this. READING the article is usually how I determine if the writer deserves my trust. The trust that is mine to hand out or withhold.

What does a Pangram score do to help me in this?

Nobody Owes Me Anything

Two things about Pangram on Substack annoy me.

To rate the AI content of my article, my article must first be sent to Pangram for processing. My text, my copyrighted prose, is ingested into a third-party classifier for processing. What stops Pangram from ingesting my prose into its dataset to refine its learning?

The new Terms of Use for Substack doesn’t explicitly state that my prose won’t be used for Pangram’s dataset. Nor does the Privacy Policy.

The Terms of Use reads:

You hereby grant Substack a license to translate, modify, reproduce, and otherwise act with respect to your Posts to enable us to provide, improve, and notify you about new features within Substack.

Pangram is a feature. The license implicitly allows the send. Hmm.

On the other side, Pangram, on their data privacy page, says:

Pangram does not train generalized AI models like ChatGPT, and our AI detection technology is based off of a large, proprietary dataset that doesn’t include user submitted content.

Plus: We never share or sell your data to any third parties.

That’s a promise to Pangram’s customers. Substack is a customer, but I am not. Which means I don’t have a contractual relationship with Pangram.

I’m told that a support document on Substack says Pangram won’t use it, but that’s not what defines my working relationship with Substack.

Substack can use my prose under the Terms of Use to offer the Pangram “feature.” Which means I am unprotected because I have no standing with Pangram. Hmm.

I didn’t sign up for that.

I have options. We all have options. I can vote with my pocketbook and go elsewhere. Different experiences. I give up convenience and services and reach in my distribution. But I am not bound by blood here. I can choose.

Guilty Until Scanned

Substack framed Pangram in trust. Pangram is intended to let us, the reader, know that what is published is written by a human and not AI.

I’m back to so what?

I am AI agnostic. AI is capable of a lot of things. And given its current rate of evolution, it’s likely going to get good at emulating human writing. Good enough to fool me. But why does that matter?

Yes, I am concerned that we can take a person’s likeness from a photo or video and use AI to put them into videos. Deepfake porn is a thing where the faces of actresses are put on the bodies of porn actors. Without consent.

If they can do this, what stops them (whoever these nefarious persons are) from taking my face and creating video proof that I robbed a store at gunpoint? With Flock footage, it wouldn’t be hard to craft video evidence that I left my home, drove to an adjacent town, robbed the cigar store at gunpoint and drove home.

Such video posted in social media could and would likely damage my reputation. Even if the video was proven manufactured.

But a deepfake video is passing off a naked lie as true. It asserts that something happened. Which is different than an AI-written essay offering an idea for my consideration. It makes no claim about its making.

The first is fraud. The second is a process question being treated as fraud.

Pangram doesn’t build trust. It preemptively undermines trust.

If I use Pangram for every article on Substack before I read it, I’m effectively saying I believe that every article is suspected of being written by AI. That’s not trust. That’s distrust.

Pangram acts like the police, standing at your door, and calling you a fraud. And refuses to show you any evidence or proof. It hands you a percentage with a presumption of guilt.

The process doesn’t prove or disprove the ideas. Scanning for process is theater.

I don’t want to enter the dance with the writer and her ideas already believing her ideas suspect. I just want to read.

Because I value the ideas.

If she writes it in prose, dances an interpretive dance with raw chickens on her hands, or recites poetry banging a crystal, if the ideas presented resonate in me, strum lightly upon the strings of my humanity, or stomp violence on the head of certainty, she’s done her job.

She’s touched me, human to human. Even if AI wrote the prose, digitally attached the flailing chickens or composed half the poetic verses.

We can sit and debate the ethics of AI in creating these modes. And wrestle with the reality of being touched, not by an angel but by an amalgam of concepts and words whipped up by AI and served up by a human.

At least we’d connect. Human to human. Idea to idea. Given the state of things, that’d be an improvement.

Pangram on Substack is dumb. And maybe Substack is creating value for its investors, possibly in anticipation of going public. Or Jim Bob Blevins thought Pangram would be a good idea. Like Segways, New Coke, or Google Glass.

And whether it stays or goes, it reminds me that good things change. And bad things improve. Or stuff stays the same.

Whatever happens, I’m going to write. You?