• assembly@lemmy.world
    link
    fedilink
    English
    arrow-up
    18
    ·
    3 hours ago

    As others have stated and I will reiterate, this (to me) is all of the technical debt coming home to roost. How often have we intended to go back and refactor something once we have time? I don’t just mean in your current job, I’m talking about throughout your career. I think about code I wrote in 1999 that may still be out there and cringe at what I put together back then. The person that has taken over after me at every job probably just built on top of what works and underlying shit got buried. Of course the business should have gone back and evaluated old code but are they really doing that? There is a vast amount of technical debt out there that was secured solely by the vast amount of other technical debt. There was always too much to dig through but now AI has reduced the resources needed to dig through it all at scale. This is a whole new world and just the beginning of the ride.

  • Aeder@lemmy.world
    link
    fedilink
    English
    arrow-up
    14
    ·
    4 hours ago

    The funny thing is that one of the reports on the glasswing project found out a lot (~55%) of the vulnerabilities AI finds are just false positives and not really worth reporting so they just increase the cognitive workload of unpaid maintainers.

    • JohnEdwa@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 hours ago

      Yes, a 55% false positive rate technically means more often than not, it isn’t actually a valid vulnerability and it is wasting your time. But if you have 1000 boxes to go through while looking for a certain item, someone with a 50% false positive rate could identify two boxes for you, and it would be in one of them.

      So unless that number is “AI claims there’s a vulnerability on every other line of code you wrote”, which I kinda doubt, and is actually based on made bug/vulnerability reports, then isn’t is extremely helpful even if it’s only right half the time?

      • MrScottyTay@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        2
        ·
        2 hours ago

        Getting what I’ve seen even the actual vulnerabilities are the sort of stuff that is very very very hard to pull off, like it’s identified a .2ms window where if you input something specific, you break something and gain access. Like race conditions that even though they theoretically exist, they’re actually impossible to execute

  • Wildmimic@anarchist.nexus
    link
    fedilink
    English
    arrow-up
    44
    arrow-down
    1
    ·
    10 hours ago

    This was pretty much inevitable - there is so much insecure code out there from practically every era of computing. There’s only one answer, and that is giving developers and security experts the resources (that includes LLMs) and time they need to ensure that all this legacy code is being fixed or replaced with better structured code that is less attackable.

    It will be a wild ride for a while, and then things will slow down again when the well of not-yet-discovered exploitable bugs starts to run dry.

    • audaxdreik@pawb.social
      link
      fedilink
      English
      arrow-up
      32
      arrow-down
      1
      ·
      10 hours ago

      I’m quite staunchly anti-LLM, but code review is where I think they are most suited. Mind you, only as PART. But their ability to hold large bodies of well-structured code in context is what gives them the edge to find possible bugs and exploits a human has overlooked.

      They never should’ve been used for code creation, and in reality they’re being use on both sides of the equation going completely unmonitored or, at best, there’s a burnt out human sitting in between rubber-stamping commits because the entire pitch for this shit is to increase productivity (output) and move at a rate designed to outpace them.

      • sunbeam60@feddit.uk
        link
        fedilink
        English
        arrow-up
        13
        arrow-down
        1
        ·
        7 hours ago

        In my experience, AI writes better code (and tests) than most developers and much worse architecture than a senior engineer.

        I suspect the architecture is fine, really, but AI is so comfortable juggling 19 balls that it’s architecture never needs simplification and clarity. But as a human reading it, it’s just a mess and humans need much more coherence and encapsulation.

        • moseschrute@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 hour ago

          I feel like AI writes good code but then every once in a while it does the absolute dumbest thing in the world. And you have to be really vigilant so you can’t it doing dumb shit.

        • log@lemmy.blahaj.zone
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          1
          ·
          4 hours ago

          Then most developers you know are subpar. Frontier models routinely write far worse code than I. And I am only a moderately decent dev.

          I have even seen llm code generated with a frontier model that blew straight through array boundaries without a worry in the world.

          They can be useful at work, especially for tech I know little about, and as an additional set of eyes to avoid easy mistakes like off by ones, or doing wrong pointer arithmetic, but they are pretty abysmal at code generation.

          • Taleya@aussie.zone
            link
            fedilink
            English
            arrow-up
            1
            ·
            53 seconds ago

            Nononono “tech you know little about” means you don’t know enough to see the holes it’s making

          • red_tomato@lemmy.world
            link
            fedilink
            English
            arrow-up
            2
            ·
            51 minutes ago

            I thought the same earlier this year. Opus 4.8 mostly produced garbage code in my eyes. Then Opus 5.5 came and now it can routinely produce about the same quality as me but much faster.

  • Aceticon@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    18
    ·
    11 hours ago

    AI Is Breaking the Entire Structure of Software That Powers the World

    On both sides even - both by better automating brute force hacking strategies AND by generating weakness-riddled code for network-facing software.

    What a great time to be alive!

    • sunbeam60@feddit.uk
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      7
      ·
      7 hours ago

      I actually think the code AI writes is much more hardened than a human’s. But the architecture that it creates is incredibly hard to reason about and feels very brittle and prone to exploitation.

      • Aceticon@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        7
        arrow-down
        1
        ·
        edit-2
        6 hours ago

        I think you’re confusing AI being better at code hardenning with modern server-side libraries for network-facing software being designed to be idiot-proof and stopping things like SQL injection via parameters - the code is safer because over the last 3 decades the libraries were made idiot-proof, not because the AI is any better than the idiots.

        The automated parrot which does things like putting keys and other access credentials plaintext in GitHub only makes code (and just code, it can’t really do design properly) as hardenned as the average found in open repositories, which includes amongst other things simplified examples in Stack Overflow (which probably explains the GitHub plaintext credentials thing)

        Most weakness for hacking nowadays are either the traditional social hacking or they’re interactions between multiple systems which haven’t been accounted for (or, even better, which were introduced when one of those systems changed but not the other), the former having nothing to do with AI and all to do with users giving access creditals to anybody who sounds “confidently in charge”, whilst the latter being made much more likely by what you yourself pointed out as a problem of AI.

  • shirro@aussie.zone
    link
    fedilink
    English
    arrow-up
    62
    arrow-down
    1
    ·
    13 hours ago

    Don’t believe the hype.

    The truth is nobody ever has time to test things thoroughly. Proprietary software has to ship early due to unrealistic deadlines. Bad specs, client doesn’t know what the fuck they want. Open source dev doing weekend hobby coding suddenly getting pressured by billion dollar corps for features while trying to balance mental health, family, pay bills etc.

    So we take shortcuts. Ship code. Hope for the best. Better compiler warnings, linters and strict coding standards can help but only so much.

    LLMs let security researchers search for classes of bug that have been harder to find with other methods. Lots of low hanging fruit for them. Most of the vulnerablities are low impact and over hyped. Once they find them and they are fixed they aren’t going to find the same exact one again. The security stuff will get better. Unfortunately not so much careers.

    • mycodesucks@lemmy.world
      link
      fedilink
      English
      arrow-up
      20
      ·
      10 hours ago

      Nobody has time to test things thoroughly because the fastest code that looks good succeeds because we looked at “the ui is the product” not as a problem to fix but as a customer to be appeased and now we’re paying for it.

      Software engineering needs to become a licensed profession like any other with professional engineering standards for efficiency and safety. It’s long over due.

  • eicker@lemmy.worldOP
    link
    fedilink
    English
    arrow-up
    151
    arrow-down
    6
    ·
    15 hours ago

    AI isn’t breaking software by itself. It’s revealing just how much of the world runs on code held together by hope, legacy dependencies, and one exhausted maintainer. The exciting new feature is that attackers can now find those cracks for $200 while the security team is still waiting for budget approval.

    • 🌸𝓯𝓵𝓸𝔀𝓮𝓻🌸@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      9
      ·
      10 hours ago

      That was always going to happen. If one supplier offers bulletproof software but demands a NASA budget and timelines and the other one offers to do it for pocket change and promises, the latter one is chosen.

    • Buffalox@lemmy.world
      link
      fedilink
      English
      arrow-up
      39
      arrow-down
      1
      ·
      14 hours ago

      Yes it is, it is making programmers leave, and it is preventing new programmers from learning the basics of a project.
      My wife just told me she that Debian is losing some of it’s most experienced programmers, because code submissions are scanned by stupid LLM that as silly questions about the submissions that MUST be responded to.
      Disrupting maintenance absolutely breaks software, and AI is doing that all by itself.
      There aren’t even any $200 saved, as Debian is maintained by contributors.

    • yaroto98@lemmy.world
      link
      fedilink
      English
      arrow-up
      70
      arrow-down
      1
      ·
      15 hours ago

      And security by obscurity. Keeping source code secret to hide your failures.

      • 2ndDegreeOWO@lemmy.world
        link
        fedilink
        English
        arrow-up
        36
        arrow-down
        1
        ·
        15 hours ago

        Yep, in the age of AI code decompiling, secret source code isn’t going to help you much.

        • boonhet@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          8
          ·
          14 hours ago

          Eh if they can get the binaries off your server somehow, they don’t really need to decompile shit, they can just test their own instance of it

          • rogerdodger@lemmy.world
            link
            fedilink
            English
            arrow-up
            14
            ·
            13 hours ago

            Decompiling is still usefull. It’s easier to find problems in the source code than blindly probing.

          • 2ndDegreeOWO@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            ·
            13 hours ago

            A lot of places are using off-the-shelf software that attackers could just buy/download a copy of from the vendor.

    • jimmy90@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      24
      ·
      13 hours ago

      this is JAVA owned by ORACLE jesus fucking christ. AI is helping FIND BUGS because well paid well resourced humans make mistakes too.

      the luddite lemmings strike again

      • Bourff@lemmy.world
        link
        fedilink
        English
        arrow-up
        16
        arrow-down
        1
        ·
        13 hours ago

        Refusing to acknowledge that AI helps every sociopathic script kiddie do real damage to the real world doesn’t help either.

  • ArgentRaven@lemmy.world
    link
    fedilink
    English
    arrow-up
    91
    arrow-down
    1
    ·
    15 hours ago

    IT has been cut to bare bones for a long time, now. Some VP made their cost cutting bonus 10 years ago. Their successor did the same. And their successor. THIS is the result. This is our culture. Make the rich richer now no matter who suffers later.

    I work in IT security and I’ve buried many colleagues and friends that died from over-stress. Usually their hearts give out. My best friend died last December. Not that rich C suite executive, though. Now there’s no one left to work late and fix all the problems. They’ve deputized whole divisions to the patching and vulnerability teams, just to stem the bleeding so the companies don’t end up in the news. But as soon as we fixed one, another comes up. I’m not speaking in abstract terms - that was literally my Wednesday. We can’t get ahead of it. A concentrated cyber attack by AI that doesn’t get tied, bored, or blocked will eventually succeed. Our legacy code running Java dependencies on Windows 95 isn’t getting replaced because it’s not in the budget.

    I just hope they don’t go for infrastructure.

  • ominous ocelot@leminal.space
    link
    fedilink
    English
    arrow-up
    22
    ·
    13 hours ago

    It is already broken. Just the cracks come to light.

    Waiting for the reveal of the first serious bugs in AMDs and Intel’s management engines. Those tiny OS running on their own core next next to the user’s OS. Or the firmware of the network cards. This will be a fun day for users of older PCs without the chance to receive patches.

  • Wakko@hell.cloud
    link
    fedilink
    English
    arrow-up
    11
    arrow-down
    11
    ·
    6 hours ago

    It never needed to be like this.

    AI may be exposing and exploiting the world’s tech debt. But AI is also the solution to the problem, too.

    LLMs now produce code fast enough to make 100% test coverage achievable for every software project.

    There are now zero excuses for having under-tested code in production.

      • Wakko@hell.cloud
        link
        fedilink
        English
        arrow-up
        2
        ·
        21 minutes ago

        After having been in the tech industry a couple decades, I have experienced a whole lot of people who get very grumpy when they encounter the truth. People like to lash out when they encounter information that challenges their deeply-held beliefs. Cognitive dissonance is often expressed in funny ways.

        What you’re seeing is the kind of large-scale existential crises that only once-in-a-generation technologies can cause. A whole industry of individuals who built their livelihoods on being semi-clever for a living are now being outpaced by semi-competently operated LLMs. Nobody is ever happy to find themselves confronted with adapt-or-die circumstances.

        And before anybody objects to my use of the phrase “semi-clever” - we wouldn’t be witnessing a vulnpocalypse if the situation were any different. For better or worse, the incompetence on display by the leadership of the US Federal Government isn’t unique. Most middle-management positions are filled with knuckleheads barely more functional than the current cabinet.

        Americans working in the tech sector have been short-changing everybody for so long that “quality craftsmanship” is treated as a four-letter word, uttered in front of your boss only if you’ve already secured a position elsewhere.

        All of the epithets I hear leveled at AI Slop are criticisms I saw directed towards incompetent coders on websites like TheDailyWTF over a decade ago. The fact is, LLMs have gotten so good at generating code in the last two years that the entire concept of corporate-managed software services is in the process of dying off - its corpse being cannibalized by millions of LLM-powered startups staffed by single-digits worth of humans.

        What we do with these tools is a choice.

    • NotAnonymousAtAll@feddit.org
      link
      fedilink
      English
      arrow-up
      13
      arrow-down
      2
      ·
      5 hours ago

      100 % test coverage and 100 % correctness are very different things, especially if the tests are auto-generated by LLMs at a rate that is impossible to properly review.

      • Wakko@hell.cloud
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        7
        ·
        5 hours ago

        Every coder trying to justify their jobs brings that up. And they’re all wrong.

        You don’t do 100% correctness already. In terms of correctness, LLMs generating code with 100% coverage will run circles around whatever you do write today. Demonstrably.

        You don’t have the winning argument you think you do. You may want to re-read the parable of John Henry.

  • Zeus@feddit.org
    link
    fedilink
    English
    arrow-up
    10
    arrow-down
    4
    ·
    14 hours ago

    This is why you need strict quality control whenever you have AI making things.

    Cybersecurity experts need to thoroughly examine all the code and how the software works…or doesn’t work and help fix it

    Then you need ethical hackers to test the security and you need the cybersecurity experts to fix the problems the ethical hackers find.

    it’s boring, grueling work but if you don’t do it everything falls apart

    • TheMightyCat@ani.social
      link
      fedilink
      English
      arrow-up
      11
      arrow-down
      1
      ·
      14 hours ago

      This is why you need strict quality control whenever you have AI making things.

      The article is about AI finding exploits.

      • mangofeet@fedia.io
        link
        fedilink
        arrow-up
        3
        arrow-down
        1
        ·
        14 hours ago

        sure, but you can’t just patch shit Willy nilly without also breaking things.

        doing things correctly takes time and money, neither of which we have time for when we have the poor, defenseless, baby bird shareholder to take of!!

        • TheMightyCat@ani.social
          link
          fedilink
          English
          arrow-up
          2
          ·
          14 hours ago

          I mean i agree that proper quality control and review is needed.

          Its just that the conclusion Zeus takes from this article is not really corroborated by the article.

      • FaceDeer@fedia.io
        link
        fedilink
        arrow-up
        3
        arrow-down
        3
        ·
        14 hours ago

        It’s funny how the narrative until this point has been that AI is too crappy at coding to be useful.

        • Maestro@fedia.io
          link
          fedilink
          arrow-up
          16
          arrow-down
          1
          ·
          14 hours ago

          In my experience, AI is really good at finding problems because of the large context window and speed. But it is really crappy at writing good code, because it can’t distinguished good code from bad. The way AI writes code is just to generate stuff and fix problems until all problems are gone. That is a terrible way to develop.

          • Awesome Lowlander@cozyver.se
            link
            fedilink
            English
            arrow-up
            4
            ·
            11 hours ago

            generate stuff and fix problems until all problems are gone. That is a terrible way to develop.

            True, but let’s be honest, that’s also how a LOT of human programmers do it.

            • Maestro@fedia.io
              link
              fedilink
              arrow-up
              2
              ·
              6 hours ago

              Yes, but all the guardrails we have to keep poor developers in line start to fail when you scale it up with AI