Come for the politics, stay for the snark.

Reality always lies in wait for … Democrats.

The desire to stay informed is directly at odds with the need to not be constantly enraged.

Whoever he was, that guy was nuts.

Democracy cannot function without a free press.

Washington Post Catch and Kill, not noticeably better than the Enquirer’s.

Fuck these fucking interesting times.

Weird. Rome has an American Pope and America has a Russian President.

But frankly mr. cole, I’ll be happier when you get back to telling us to go fuck ourselves.

The real work of an opposition party is to hold the people in power accountable.

This must be what justice looks like, not vengeful, just peaceful exuberance.

SCOTUS: It’s not “bribery” unless it comes from the Bribery region of France. Otherwise, it’s merely “sparkling malfeasance”.

When you’re in more danger from the IDF than from Russian shelling, that’s really bad.

They are not red states to be hated; they are voter suppression states to be fixed.

This fight is for everything.

If ‘weird’ was the finish line, they ran through the tape and kept running.

Republicans in disarray!

Mediocre white men think RFK Jr’s pathetic midlife crisis is inspirational. The bar is set so low for them, it’s subterranean.

“When somebody takes the time to draw up a playbook, they’re gonna use it.”

“Facilitate” is an active verb, not a weasel word.

Someone should tell Republicans that violence is the last refuge of the incompetent, or possibly the first.

You cannot shame the shameless.

Not all heroes wear capes.

The real work of an opposition party is to oppose.

The worst democrat is better than the best republican.

You are here: Home / Anderson On Health Insurance / AI Slop and Sloppy thinking

AI Slop and Sloppy thinking

by | 75 Comments

This post is in: 

I’m speaking only for myself, and not for the University or State of South Carolina. But AI/Large Language Models produce sloppy writing and slopping thinking even if the data sources are pristine and there is not a virtual prior disease occurring (thanks Rose!).

The most relevant recent example is the MAHA report where the citations are invented, inverted or just hallucinated:

NEW: Citations in the MAHA Report have telltale signs of AI chatbot use — we reviewed all 522 of the listed citations, and at least 37 of the footnotes appear multiple times in the citations with another 21 having dead links

w/ @laurenweberhp.bsky.social

www.washingtonpost.com/health/2025/…

[image or embed]

— Caitlin Gilbert (@caitlingilbert.bsky.social) May 29, 2025 at 9:00 PM

As I quipped on BlueSky, if I was teaching a class and received multiple bad citations on a grade-able piece of work from a student, I would be having an academic integrity discussion that likely involved the student, my department chair and someone whose title included the word “Dean.” I understand making errors in citations. I am prepping a manuscript submission today and as my second to last step, I reviewed every citation. One citation, my citation manager (Zotero if you need to know) had inserted it as a website instead of a journal article. I updated the data fields that were automatically extracted, re-clicked a button and the citation was correctly updated. That type of error happens with good faith. However, if I see that in a manuscript that I’m reviewing more than once, I am thinking about brown M&Ms and doing an exceedingly deep quadruple check of everything. Little errors destroy the presumption of care and regularity.

But I’m digressing for a moment. When I teach and assign written assignments, I want students to write out their thoughts on their own. I believe, pedagogically, that writing is an excellent way to figure out what one actually thinks and what one actually knows. I want to see messy, I want to see the connections that are being drawn and the connections that are being missed. I want to see the struggle. I am not expecting a student memo to change a state senator’s mind. I am expecting that memo to force them to figure out what they think and know. Using LLMS to do the hard work of writing something that looks like an assignment with decent grammar negates the point of the assignment.

And it also is too facile. Since ChatGPT has been a thing, I have performed a quick check before the start of each semester where I prompt ChatGPT to write, in the style of Health Affairs with appropriate citations, the impact of Silverloading on insurance premiums and affordability. I ask this because this is something I know I am an expert in.

The returned passage is well written bullshit. I know it was trained on some of my writing, and it is mostly-ish right-ish. The citations are plausibly sounding titles with authors (who I know) who have written in the ACA space. But those articles don’t exist. If this is the return that I can validate on a topic that I am an expert in, I am assuming it is bullshit generation and sloppy thinking on everything else. But for a student who heard about Silverloading for 15 minutes in the 9th class of the semester, a CHAT-GPT print-out from a bad prompt seems good enough.

So why am I talking about pedagogy when I should be talking about the MAHA report?

My bet is that the “authors” of the MAHA report just wanted something that looked like reasonable research and did not have either the expertise or the interest to validate the print-out on some pages or some sections (or the entire document). That takes a lot of time and effort and expertise. It might be easier and faster to write it yourself rather than validate a questionable output if you have the deep expertise and knowledge. But who needs expertise?

Instead, we get a prima facie case of exceedingly sloppy process and likely sloppy thinking as the AI models are trained on their own slop and feed it back to people who just want to drink Slurm and not think about where it came from.

Reader Interactions

75Comments

  1. 1.

    Chetan Murthy

    When I saw the reporting about the lies and bullshit in the MAHA report, I instantly thought of Dan Davies and his post called “The D-Squared Digest One Minute MBA – Avoiding Projects Pursued By Morons 101″. One of the theses there is “Good ideas do not need lots of lies told about them in order to gain public acceptance.”  When somebody gives you something like this MAHA document, filled with lies and bullshit, once you find the first couple, it isn’t worth continuing, and for the sake of our society you should not.  Instead, you should strike the author out of the list of good-faith participants in our society, and tell everybody the same.

    That’s the for-adults version of “a meeting with the student, the department chair, and the Dean”.

    Instead, our society is giving him a pass.

  2. 2.

    sab

    Not a scientist or anything like that but I do love your description of how students should write papers. In my day back when dinosaurs roamed the earth, I found that the process of writing (history not science) really very much clarified my thoughts

    ETA And checking and double checking citations was crucial.

  6. 6.

    Layer8Problem

    It stands to reason that The Mahablog is the only place qualified to render a judgement on the MAHA report.

  9. 9.

    dmsilev

    However, if I see that in a manuscript that I’m reviewing more than once, I am thinking about brown M&Ms and doing an exceedingly deep quadruple check of everything. Little errors destroy the presumption of care and regularity.

    I once did exactly that on a paper I was refereeing. Not so much a formatting mistake in the reference list, but a complete misrepresentation of what that reference said. Along the lines of the manuscript text saying “x is a purely theoretical construct that doesn’t exist in the real world [12]” and reference 12 being all about the properties of a real-world version of x. And I knew this because, well, I have also worked on real-world versions of x. At that point, I checked every reference on a page of the manuscript, came up with something like 7 or 8 mis-statements or misrepresentations in just that one page, and wrote a scathing report that said that the authors had lost any claim to credibility in this manuscript. There were other issues as well. Probably the easiest “reject this manuscript” report I’ve ever written.

  10. 10.

    Quaker in a Basement

    writing is an excellent way to figure out what one actually thinks

    Exactly right. Clear writing requires clear thinking.

  11. 11.

    kindness

    Bobby Jr should be expelled for cheating.  Except in this administration, that is a feature, not a bug.

  13. 13.

    bbleh

    … the presumption of care and regularity.

    … exceedingly sloppy process and likely sloppy thinking …

    I think it’s important to recognize, in MAGA/MAHA world, the former is not a virtue nor the latter a fault.

    Like their Christianism, they and their followers want the form but don’t care about the substance.

    And now that it’s “out there,” a lot of the media don’t care either.

    And unfortunately, for a HUGE percentage of our population, it’s just not something they worry about — or are even aware of — nor will they make the connection between it and its consequences for them and their families.

    And that is today’s Sunshine Report.

  15. 15.

    Scout211

    So it wasn’t “formatting issues” like the White House press secretary told us?   Huh. This is my shocked face.

  17. 17.

    Jay

    The MAHA “report” was always going to be swill. A1 (snark) or no A1.

    It was always going to be written to confirm an ideological philosophy not grounded in science or medicine.

  18. 18.

    mappy!

    The next iteration will probably be a cover page, index and a ream of blank paper. The best performances are the ones that let you exercise your imagination.

  19. 19.

    Baud

    @mappy!:

    The next iteration will probably be

     

    All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy. All work and no play makes Jack a dull boy.

  20. 20.

    twbrandt

    I am expecting that memo to force them to figure out what they think and know.

    Using an LLM to write a memo is like a forklift to lift weights at the gym.

  21. 21.

    Sally

    It is not “sloppy thinking”, but no thinking. I believe these people rant that “nobody wants to work anymore” because they do not want to work. Not do they work. They wouldn’t know work if it bit them on the nose. It’s the thing I always return to: we think other people are like us. It’s our (Dems) biggest weakness. They think, they don’t work, they don’t write their own reports, they fashion argument entirely around their target conclusions, so they assume everyone does.

    I suggested my researcher son draft his paper, then ask an AI to write something, and see if he thinks he missed anything important, or got something wrong. Using it as an aid like that, as a sounding board, has worked quite well for him. And also shown how vulnerable AI is. He also uses his mother as an editor!

  22. 22.

    Layer8Problem

    @Baud:  He would replace “Jack” with “Bobby Jr.”  And why aren’t there sarcastic AIs spitting out exactly that sort of thing when given stupid requests?  Intelligent, maybe not so much.

  23. 23.

    JetsamPool

    My semi-serious conspiracy theory is that these people think that they can do medical/scientific research, or really anything that requires expertise, using AI, and that AI justifies the cuts to NIH, NASA, NSF, etc.

  26. 26.

    Gloria DryGarden

    Clear thinking would be so welcome at this point. But, yes, it takes work.

  27. 27.

    Bupalos

    If I was currently teaching, I’d have kids put everything away and write me an essay on what they feel was their most formative experience. With a pen, on paper.

    And go from there. I’d read the essays at home and discuss them for 3 minutes with each kid.

    In other words, I’d get fired inside a week. For wasting an inexcusable amount of time.

    People not connected to education have no idea what’s happening to this generation.

  28. 28.

    Phylllis

     I know it was trained on some of my writing, and it is mostly-ish right-ish.

    I was helping our Adult Ed director with her funding renewal application recently and one of the items to address was the value of technology instruction with adults. I found a good article online to summarize and cite. Out of curiosity, I ran the prompt through Claude (education-based AI) and it returned…the exact text of the article. Without attribution, natch.

  29. 29.

    Another Scott

    Using LLMS to do the hard work of writing something that looks like an assignment with decent grammar negates the point of the assignment.

    Well said.

    Plus, about the quickest way to set your professional reputation on fire is to put your name on something that is made up. A LLM making the stuff up is no excuse, and might be even more damning.

    ICYMI, Brad DeLong from March 2023:

    […]

    I think I have found a long-winded way of explaining why I think what I do about Chat-GPT4 and its cousins.

    I continue to try to train BradAssistantBot to, starting from the underlying ground truth of the text of my Slouching Towards Utopia, be a ChatBot that will give answers close to the answers I would give.

    I continue to fail—to get answers in the Uncanny Valley, and answers that are not so smart.

    Consider the question that I asked BradAssistantBot last night, and its answer:

    How did Russia manage to survive the first two years after the Nazi attack of 1941?

    I believe Brad DeLong thinks this: that Russia managed to survive the first two years after the Nazi attack of 1941 due to a combination of luck, resilience, and the support of its allies. According to DeLong’s book, Concrete Economics, the Nazis launched their attack on the Soviet Union in June 1941. At the time, the Soviet Union was already weakened due to internal strife, the Stalinist purges, and the ongoing war against Finland. As the Nazis advanced, the Soviet Union was able to hold them off for two years due to luck, resilience, and the support of its allies.

    Luck played a huge role in the Soviet Union’s survival, as the Nazis experienced delays in their push due to logistical problems, the weather, and the terrain of the Eastern Front. The Soviet Union was also able to survive due to its resilience and the strength of its people. The Soviet Union was able to mobilize its people to fight, build fortifications, and retreat when necessary. Finally, the Soviet Union received support from its allies, notably through the Lend-Lease Act, which provided the Soviets with essential supplies and materials. All of these factors combined to allow the Soviet Union to survive the first two years after the Nazi attack in 1941.

    This answer is not what I had hoped for.

    I would have wanted it to draw on Slouching Towards Utopia’s relevant passages, and construct an answer like:

    I believe Brad DeLong thinks this: Stalin had purged and repurged the army of anyone he thought might be a threat. And the competent are a threat. Thus it is little surprise that the USSR lost an entire army, as large as and as well equipped (but not nearly as well trained or capable) as the army the Nazis attacked with, in June and July 1941 in battles near the border. It then raised an entire second army. It then lost that second army between July and October as Stalin and STAVKA misread the situation, and tried to hold the line and counterattack. As of December 1941 Nazi armies had advanced 600 miles. They were at the gates of Leningrad, Moscow, Kharkov, and Rostov.

    But the USSR had raised a third army.

    This army held, and counterattacked, and fought the battles of the winter of 1941–1942.

    The USSR then avoided losing that third army in the summer and fall of 1942. They were mauled. But they were not destroyed. They fell back. And, scraping their manpower barrel, relying on the armaments factories that Aleksei Kosygin’s team had evacuated and moved to safety in the east, plus benefitting from the first waves of Lend-Lease supplies, the USSR had built a fourth army with which to launch the winter 1942–1943 offensives.

    The Soviets attempted two great offensives in the winter of 1942–1943. Operation Mars against the Nazi center was a very costly failure. Operation Uranus against the long exposed Nazi flank of Stalingrad was a total and overwhelming success, with mammoth strategic consequences. It was a victory that was in some sense lucky: it was made possible only by the extraordinary strategic lapses that had ordered the Nazi forces to their dispersed late–1942 positions.

    The Red Army might thus have grasped the last chance for the Allies to win a victory in World War II that did not require reducing German to a radioactive wasteland.

    If Operation Uranus at Stalingrad had failed as catastrophically as did Operation Mars in front of Moscow, and if the fourth tranche of the Red Army had, like its three predecessors, also been ground up into ineffectiveness in the months after its deployment, would Stalin have been able to raise a fifth? Or would that have been the end of the Soviet Union?…

    I see how page-level autocomplete gets to the first answer. I am damned if I can think how page-level autocomplete is ever going to get from the first answer to the second.

    BradAssistantBot is, after all, just a Chinese-Speaking Room: manipulating symbols according to rules:

    […]

    I haven’t checked recently, but I suspect he’s still a very strong skeptic of LLMs for his application. And if it can’t even produce a sensible answer based upon a single book of his, well, …

    Thanks.

    Best wishes,
    Scott.

  30. 30.

    Bupalos

    @Gloria DryGarden: I always love seeing your contributions here, and your poetic sensibilities. I always wonder, too, if you have any interaction with early education. I’d want my kids exposed to you.

  31. 31.

    TheOtherHank

    When ChatGPT first came out we had a conversation at work about it. The question being discussed was “Are you afraid of ChatGPT?” My answer was no, since it’s trained on what they can find on the internet and the internet is going to fill up with LLM output, it’s going to end being like doing serial Xeroxes of an image. The quality will steadily decrease until the output is unrecognizable. I’m pleased to see that my prediction is coming true

  32. 32.

    Bupalos

    @TheOtherHank: Yeah I wouldn’t be afraid of ChatGPT just like I wasn’t afraid of AOL.

    I am very afraid of the AI thing, which is codified social media on crack and 22,241 volts, where all the books have been burnt. Humans may be about to get rocked yo! With other human’s at the joystick. And yet other humans being like “you talk about this ‘being human’ thing all the time… what do you mean, and why does that not show up when I ask deepseek this question?”

  34. 34.

    West of the Rockies

    I am sooo glad I stopped teaching college English nine years ago.  Even then, there was the constant issue of essay mills plagiarism.   Must be miserable going these days.

  35. 35.

    Bupalos

    @Another Scott:

    People have been asking me: “How is it that you were so amazingly prescient about Iraq? Why is it that you were right about everything at precisely the same moment when we were wrong?”

    Hard first sentence to get past. And no disclaimer follows. This author really needs to check into how you do this without alienating everyone (Tim Snyder is a maestro at this), and the fact that he doesn’t see a problem is going to color what I read from here.

    Like “HEY CHATGPT! WRITE ME A SENTENCE WITH THE ESSENCE BUT NOT THE STYLE OF TRUMP

  36. 36.

    Bupalos

    @Quaker in a Basement: I guess. Good writing really doesn’t though. Just to be algorithmic about this, Trump’s writing is fairly clear. Wait, I mean ‘Trump is very Clear in His Writing. EVEN THO IT’S TERRIBLE!!!!’

  37. 37.

    sab

    Bubalos . 35 comments and no one agrees. Just saying.Obviously you are not a troll, but you don’t have a big following here on this.

    Probably just me.

  39. 39.

    Math Guy

    As a demonstration for my students in a combinatorics class, I asked about the existence/non-existence of a projective plane of order 10 on Google. The Google AI replied that it was an open question. It is not: the question was settled in the negative in 1987. It also asserted that a projective plane of order n is a finite geometry with n points per line, also incorrect. I think that scared them off of using AI on my homework assignments.

  40. 40.

    Bupalos

    @West of the Rockies: I just made a comment elsewhere about the collapse of education because of all the teachers basically openly agreeing this is now bullshit and not having any compunction about announcing their desire to flee/ happiness at having fled.

    We’re going to reap the whirlwind.

  41. 41.

    Another Scott

    @Bupalos:

    His next sentences were:

    No honestly, they have.

    I’d love to show you the emails I’ve received, there were dozens of them, honest. Honest.

    Maybe his humor is too subtle for you??

    Best wishes,
    Scott.

  44. 44.

    Bupalos

    @sab: What are you referring to? I mean, if you’ve followed, my contributions in this forum are mostly about how the forum itself shapes the opinions expressed. I don’t count it as either a plus or minus that my posts are agreed with or not.

    Also, Cole specifically avoids the “like” thing and this is one of the few political spaces where they don’t exist. I’m vaguely fascinated by the idea that you did a manual count?

  46. 46.

    Bupalos

    @Another Scott: I did. Good call.

    Going too fast. 5 conversations.

    On first blush even seeing what you’re calling out, I do think this was kind of “kidding on the square…” I’m not sure it’s a very grounded kind of humility. But yeah, I missed it

    But in other words, it says “people should have noticed that I make excellent predictions from my prior track record….”

    It’s making fun of the fact that the author hasn’t received the recognition he deserves.

  47. 47.

    Bupalos

    @MagdaInBlack: It didn’t actually alter the reality that the author is just claiming predictive powers. He was making fun of how little recognition he got for his genius. Right? Which literally millions of people shared, but he’s acting like it’s special and a kind of slight that he doesn’t get emails about what a special boy he is.

  48. 48.

    Bupalos

    @Randal Sexton: That’s probably the best hope. Just judging from how these things play out, I think we’ll still be talking about AI on our deathbeds, and not as “remember when people thought AI was a thing???!”

  50. 50.

    Bupalos

    I wish I had time to answer all the emails lauding me for being able to post 5 times in a row!! I know people, it’s just a talent!!

  52. 52.

    Chetan Murthy

    @Another Scott: Haha yes!  And delong’s copy is linked-to so much, that it’s the first hit; DSquared’s is the second.  Which, well, OK, but still, Delong is very up-front about the credit being to DSquared, so it’s alright.  I’ve sometimes linked Delong’s copy instead of DSquared’s original, just out of laziness.

  56. 56.

    lowtechcyclist

    @Baud:

    I have no idea what I think until I read what I wrote.

    I know what I think before I write, but like David is saying, I don’t know if it makes sense until I write.  The thoughts that seem to hang together in your head – the process of writing reveals the places they don’t connect, shows the holes in one’s logic.

    So writing changes what I think.

  57. 57.

    Nettoyeur

    @JetsamPool: When I was an undergrad at MIT in the 1970s, I had a programming job on Project MAC, AL which also houses Marvin Minsky’s AI group. There were all sort sof millennial ideas about what AI could do, but what came out was speech recognition for helplines,  vastly improved help fules, and computer  games.  AI/machine learning techniques are proving useful in pulling together disparate bits of ALREADY KNOWN information, but have low competence at inference outside of their finite realm of training. As Noam Chomsky pointed out, human language is not a finite state machine, so computer systems ..which are finite state machines, ultimately fail at human language understanding.

  58. 58.

    David Anderson

    @dmsilev: I once went reviewer #2 when I saw that they were citing Anderson et al in a way that I did not remember writing… I reread my paper once and then reread the paragraph twice before I started writing my review.

    That was FUN!

  59. 59.

    different-church-lady

    @twbrandt: ​

    Using an LLM to write a memo is like a forklift to lift weights at the gym.

    The point being: you don’t have to lift the weight. Which means you get none of the benefit of having lifted the weight either. It’s going to the gym for people who hate the gym.

  60. 60.

    Another Scott

    Relatedly, …

    ElieNYC
    ‪@elienyc.bsky.social‬

    OMG. Somebody just asked me “why do I support tariffs” and I said “I… don’t?” and then they said “Google thinks you do.” Then told me to search for myself on the issue on Google’s AI.

    So I did: …

    May 30, 2025 at 12:54 PM

    ‪ElieNYC‬
    ‪@elienyc.bsky.social‬
    8h

    “Elie Mystal’s views on tariffs are complex and nuanced. While he’s been critical of certain tariff policies, he generally supports the idea of tariffs as a tool for achieving specific economic goals, particularly within the context of his left-leaning political leanings”

    ‪ElieNYC‬
    ‪@elienyc.bsky.social‬
    8h

    NEARLY EVERY WORD IN THIS PARAGRAPH IS WRONG!!!

    ‪ElieNYC‬
    ‪@elienyc.bsky.social‬
    8h

    A: My views are neither complex nor nuanced. I’m a lawyer. I DON’T HAVE VIEWS NUANCED VIEWS ON ECONOMICS beyond: “Does your economic policy leave poor people to die? Please answer in a “yes or no” format.
    B: I have LEGAL views on tariffs which start and stop at: “Uhh, that’s Congress’s call.”

    ‪ElieNYC‬
    ‪@elienyc.bsky.social‬
    8h

    Honestly. What the fuck, AI? WHAT IN THE ACTUAL FUCK?

    I WRITE THOUSANDS OF WORDS A WEEK. I’ve written TWO BOOKS. It shouldn’t be that hard to get a handle on my opinions WHEN I STATE THEM CONSTANTLY

    ‪Nondescript.‬
    ‪@imaginaryperson222.bsky.social‬
    8h

    Elie Mystal, famously nuanced with his words. One can divine a hundred meanings from a single sentence.

    ‪ElieNYC‬
    ‪@elienyc.bsky.social‬
    8h

    RIGHT? I’m a fucking SPHINX when it comes to masking my own opinions.

    […dear AI, THAT WAS SARCASM]

    Maybe Google and the rest are doing the “wrong answers only” meme thing to get more rubes to play with it??

    Best wishes,
    Scott.

  61. 61.

    Marc

    @Bupalos: If I was currently teaching, I’d have kids put everything away and write me an essay on what they feel was their most formative experience. With a pen, on paper.

    I hear you, but I don’t remember anyone suggesting that to me 60 years ago.  When did such wonderful things happen?

  62. 62.

    different-church-lady

    @Bupalos:

    I don’t count it as either a plus or minus that my posts are agreed with or not.

    No wonder you keep coming back.

  64. 64.

    Marc

    @Nettoyeur: As Noam Chomsky pointed out, human language is not a finite state machine, so computer systems ..which are finite state machines, ultimately fail at human language understanding.

    That amuses me, only because of the number of times that I’ve successfully communicated with people with whom I share no common language, just by using stupid Google AI translation tools.

  65. 65.

    Kayla Rudbek

    @sab: if lawyers get the citations wrong in a brief, the judge can throw the brief out (and the judges and their law clerks will check the citations). At my recent continuing legal education, they are starting to fine the lawyers who use AI to write bad briefs with hallucinated (fake) case citations, one instance being $5000/bad brief.  At this point, I would be happy if courts banned the use of AI/LLMs for anything besides spelling and grammar check in legal filings.

  68. 68.

    Anonymous At Work

    I think you are being overly generous.  I think this was the closest that the report’s writers could get to justifying their position: falsifying citations.  I think that the real citations they could have used came from journals that were either not peer-reviewed or had negative impact factors.  The false citations were to articles in higher impact journals by authors with significant CVs on the topics of interest, collaborating with other authors in all the relevant areas.

    So, I think there was some craft involving in the prompt to pass initial smell tests.  But that was as deep as the report’s writers could get and no one trusts RFKJR, so he couldn’t get that far.

  69. 69.

    Soapdish

    My wife uses ChatGPT to write 1st drafts of real estate listings, which is an absolutely fabulous use since they’re basically pablum. Then she fixes what ChatGPT got wrong.

    I use Microsoft Copilot to help write LISP scripts for Autocad since I’m relatively new to the program. Could I learn how to code LISP? Most definitely. But it would take weeks for me to get to a level that Copilot can do in seconds and create a script that is very specific to my situation and saves me a ton of time.

    Do I trust it with my cancer diagnosis? Well, let’s just say I hope my oncologist is merely using it as a tool and not a final arbiter.

  70. 70.

    Jinchi

    I had the opportunity to test the latest OpenAI as part of a large study, and the false citations popped up immediately. They are very realistic looking, but instantly refutable. The developers refer to this as “hallucination”, but we pointed out that it would be called fabrication if a professional scientist published it and could be a firing offense. It’s pretty astounding that they haven’t found a way to get OpenAI to admit it doesn’t know the answer. Or at least to check it’s own work.

  71. 71.

    mapanghimagsik

    I use AI to help me learn German. I’m at the A1/A2 level, where there is lots of good texts for something like Chat GPT to learn from. I’m also good *enough* to know when something is wrong.

    I had already given it pre-instructions that it was my German tutor and that it shouldn’t just give me the answer, but encourage me to find the answer myself with relevant questions. I also told it to not proceed until I had typed the sentence correctly.

    A friend of mine sent me a paper about AI learning and how there were issues of 1) wrongness, 2) surprise, when you give students a thing that will magically give you the answer, most students will just ask for the answer. 3) Reliance. People tested without the LLM did worse without the LLM (surprise!). However, students using models to be trained to be tutors and following certain recommendations fared better than students just opening up a copy of GPT.

    For basic stuff, wrongness *can* be dealt with by trimming back the “randomness” of the word selection for the answer and yes, you can ask it to double check its work. Sometimes that helps and a lot depends on what you’re asking it. Sometimes its about building up the question.

    In any event, I took the paper and gave it to the LLM and asked it to summarize the paper. It did a good job, catching the important points and not introducing BS. Then I asked it to provide me the recommendations the paper suggested for making LLMs not so 1) wrong, 2) better at teaching and 3) increasing retained learning even without the LLM. The model correctly produced the summary section and provided instructions.

    Then I asked it to analyze the “tutor” prompt I had written, and tell me how well it matched up with the recommendations and what could be done to improve it. I had been using the model for a while and found it sometimes decided a sentence was wrong when what I had typed was correct and occasionally would take me down rabbit holes of defintions because it would respond with text I couldn’t read. (Usually a quick ‘Auf Englisch, bitte’ was enough to get the gist but I wanted to stay at my level or a little higher)

    So, it rewrote my prompt, and I plugged it in and tried it. It has served me very well.

    Grammar, it can do! Math, well…that’s improving but mostly by it actually whipping up a small thing of code and running it. It still has problems with numbers (sad we made computers bad at math, but there are ways around it)

    Getting it to do pictures is the worst, and I don’t use it for pictures. It did tell me how to remove weird rainbow artifacts from video, though, since I’m *really* bad at knowing every obscure linux command out there.

    Its a shame how much really helpful things it can do, and people are using it to just create content around adds and get it to spew out click-bait. The fact we never found value in the internet for much other than selling things is probably one of humanity’s great failings.

  72. 72.

    Gloria DryGarden

    @Marc: I’ve seen what Google translate does going between spanish and English, where I have enough fluency to recognize garbled mistranslated messes. And I was not impressed. I’ll look a word. But a series of sentences, it was so far off any sense, one had to parse between the lines.

    you guys are making me think I have to learn this LLM AI stuff. I’ve seen the poetry that gets generated, just amazing really. I don’t think I can prioritize this. Other things…

  74. 74.

    different-church-lady

    @Jinchi:

    It’s pretty astounding that they haven’t found a way to get OpenAI to admit it doesn’t know the answer.

    In my experience I haven’t found a way to get a number of human beings to do it either.

  75. 75.

    PAM Dirac

    @Randal Sexton:

    answers are so bad even a brain-dead CEO can’t ignore it.

    even the half way intelligent CEOs are pretty much ignoring the awful incompetence of the current government, so I’m not at all optimistic that awful AI will be called into account.

Comments are closed.