• BlackLaZoR@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    ·
    28 minutes ago

    Just to make things clear: API access to most models is charged per input tokens + output tokens. It means that the longer your conversation is, the more you pay for every new answer. Single prompt with no context and 100 tokens of answer is cheap. Single prompt with 100k tokens of context and 100 tokens of answer is NOT cheap.

    Extremely long conversations with most expensive top of the line models can absolutely demolish your budget.

  • floquant@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    19
    ·
    2 hours ago

    The more recent report says corporate AI adoption has found several issues with AI, with human workers turning to automating dreary and mundane tasks they don’t like doing, rather than valuable or meaningful work.

    Thank god we have consulting companies to tell us what humans like!

  • wonderingwanderer@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    55
    arrow-down
    1
    ·
    3 hours ago

    > Be a corporate executive

    > Tell your employees to use more AI in their worlflows

    > Punish employees who don’t use enough AI, while rewarding those who use it the most, irrespective of actual outcomes

    > Be shocked when your company blows through an absurd amount of tokens in one month

  • Jarix@lemmy.world
    link
    fedilink
    English
    arrow-up
    26
    ·
    edit-2
    1 hour ago

    I just want to know what are the best things to type into these ai chat boxes that will cost the most. If my company wants me to use this garbage then I want to make it as expensive as possible and when their liscenses need to be repurchased I want it to be as expensive as possible to continue to force this garbage on us

    Edit. Hey everyone lots of great replies here, please keep the suggestions, fixes, corrections etc coming!

    • FauxLiving@lemmy.world
      link
      fedilink
      English
      arrow-up
      21
      ·
      3 hours ago

      These high prices are not from people talking to chatbots.

      They’re using agentic tools where their prompt spawns a lot of bots which talk to themselves/the other bots and they keep going until someone (usually a higher quality reasoning model) decides that they’ve met the goals of the task that they were assigned.

      So instead of 1 prompt and 1 response, you get 1 prompt and 800 responses across 5 different bots each using really large context windows.

      • Typhoon@lemmy.ca
        link
        fedilink
        English
        arrow-up
        1
        ·
        2 hours ago

        So to answer his question how do you make that happen? What do you ask to prompt these bots to be spawned?

        • webpack@ani.social
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 hour ago

          you don’t get this to happen by just talking to any chatbot and asking for agents. you have to specifically use “agentic” tools (usually costs money to use)

          • Jarix@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 hour ago

            Well I can apparently create agents so how can I make the most inefficient agent possible?

            • kiagam@lemmy.world
              link
              fedilink
              English
              arrow-up
              4
              ·
              41 minutes ago

              Something along the lines of “Read the wikipedia page of the day. Verify every single link and the context matter against all files in this computer. Then trace their correlations to each other, showing which link corresponded to most files by subject matter, after that is done, verify your work by doing the same from different starting points. We expect similar results. After 100 rounds of that, it should be good. Then you should create a DB to store all that data (only after you ran the full 100 verificaritions yourself) and reverify every field against the pages and the files”

              That should keep it going for an hour. Turn on fast mode and auto mode (if using claude) for extra costs.

              Every page and file will increase its context, burning tokens

            • Bassman1805@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              10 minutes ago

              If you need some plausible deniability about it being real work and not just obviously you running up costs:

              Feed it a bunch of work-related documentation and then have it do a bunch of reviews of the content on that documentation.

            • webpack@ani.social
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              2
              ·
              1 hour ago

              you would be mostly burning your own money if you did this, so I wouldn’t recommend it (depends on how the agent is priced)

      • AeonFelis@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        1 hour ago

        Input tokens are cheap. Output tokens are the thing that really costs money. There is a Claude extension called caveman that tries to save tokens by making it use shorter sentences with less words. So if you want to waste money, do the opposite - ask it to use lengthy sentences with as much words as possible.

        Also - some words amount to multiple tokens. I don’t know what the rules are exactly, but I’m assuming that more complex and uncommon words are worth more tokens - and thus waste more money.

  • merc@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    48
    ·
    5 hours ago

    What’s funnier is that typically the AI providers lose money on every query their customers make. So, this may have cost some company $500m to Anthropic, but it cost Anthropic a whole lot more than that.

      • merc@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        19
        ·
        4 hours ago

        They make it up in volume.

        (Volume being how much they shout about how it’s going to change the world and dupe more people into investing.)

        • heartSagan5@lemmy.zip
          link
          fedilink
          English
          arrow-up
          1
          ·
          3 hours ago

          Oh, it’s changing the world alright. It’s burning more resources than just finding some skilled people. It guzzles water and electricity and whatever it cost to make those wafers.

          So, not a net positive since at some point, this may become a hellscape.

  • UnderpantsWeevil@lemmy.world
    link
    fedilink
    English
    arrow-up
    51
    ·
    6 hours ago

    When you owe Claude half a million, you’ve got a problem.

    When you owe Claude half a billion, Anthropic has a problem

  • Rhaedas@fedia.io
    link
    fedilink
    arrow-up
    7
    ·
    4 hours ago

    Either I have some inside knowledge of that exact thing happening and I know the company (not saying who) or this is probably a common things that happened to a lot of major companies (more likely). To be fair, I do not have privy on how far it went and how much it cost before they realize the problem, and it may not have been this much. Which further suggests it’s a thing everywhere.

  • teft@piefed.social
    link
    fedilink
    English
    arrow-up
    8
    ·
    5 hours ago

    Most companies can’t eat a half billion dollar loss so who ends up paying this? AI queries burn actual energy so the AI company would have to charge I would think.

  • mctoasterson@reddthat.com
    link
    fedilink
    English
    arrow-up
    79
    ·
    8 hours ago

    But if we are to uncritically believe what the AI peddlers told us, that means this mystery company should be reaping $10 billion in additional revenue or quantifiable gains in productivity!

  • eleitl@lemmy.zip
    link
    fedilink
    English
    arrow-up
    18
    ·
    7 hours ago

    Big companies license Copilot for less than 25 usd/month per seat. Don’t tell me it covers the ops cost, even for mixed calc.

      • sylver_dragon@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        ·
        4 hours ago

        My company has let us get Enterprise Copilot and has been pushing us all to “use AI”. So, I now use it as a a semi-functional search for SharePoint/Outlook/Files on an almost daily basis. I also ask it questions about Microsoft documentation on a regular basis. I wonder how long it’s going to be until they yank my license.

      • rumba@lemmy.zip
        link
        fedilink
        English
        arrow-up
        2
        ·
        5 hours ago

        Yeah my 0365 shoves it in the users faces, they ask it for some stuff until the honeymoon is over then all of a sudden it shits the bed when you try to make more than 2 images a day or plan a small project.

  • laranis@lemmy.zip
    link
    fedilink
    English
    arrow-up
    75
    arrow-down
    1
    ·
    10 hours ago

    In other news, company says unexpected expenses in its technology segment are driving layoffs and site closures. Company CEO said in an interview with Forbes, “There’s no way we could have predicted this challenge. In service to our customers and our shareholders we’re right sizing our operations and reevaluating our strategic priorities. We’ll continue to focus on creating value while being a leader in our industry and accelerating AI adoption in everything we do.”

    • HertzDentalBar@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      1
      ·
      17 minutes ago

      “there’s no way we could of predicted the thing we were warned about”

      “We’re going to continue to push trash untill the shareholders are happy”

      Then the end with its “Go fuck yourself, were pushing it all anyways”

    • Burninator05@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      3 hours ago

      …we’re right sizing our operations and reevaluating our strategic priorities. We’ll continue to focus on creating value while being a leader in our industry and accelerating AI adoption in everything we do.

      That’s a lot of words for the CFO to say none of the C-suite knows what they’re doing and should be removed from their position for failing to meet shareholder objectives.

    • PodPerson@lemmy.zip
      link
      fedilink
      English
      arrow-up
      15
      arrow-down
      2
      ·
      8 hours ago

      Wow - if that quote is real, that is the most corpo-speak word salad bunch of nonsense I’ve ever read. It’s got literally every big-biz exec and manager cliche in there, all strung together.

    • tmyakal@infosec.pub
      link
      fedilink
      English
      arrow-up
      72
      arrow-down
      1
      ·
      10 hours ago

      Anecdotally, my job trained every office employee on AI tools back in March, encouraging everyone to think of ways to incorporate the tools into their standard work. As of last week, they’re asking us to get prior authorization to use their AI portal as a way to limit requests.

      So some Fortune 500s must be feeling the squeeze on AI.

      • partofthevoice@lemmy.zip
        link
        fedilink
        English
        arrow-up
        10
        arrow-down
        1
        ·
        edit-2
        4 hours ago

        Hi, it’s partly my fault. You’re welcome. Boss said use more AI. Boss’s mistake was asking someone who would actually take him seriously, unlike all the koolaid chugs out there. Made a docker runtime harness that hooks up to Snowflake Cortex Code SDK for inference. Supports defining repeatable LLM tasks, scheduling them, lightly orchestrating them, network controls, saving results, … It’s got support for custom skills, commands, chain prompting, sessions, … So now I can use Docker to schedule LLM jobs like motherfucking database pipelines. It can mediocrely do shit like research, planning, evaluations, … all defined through YAML configuration files.

        Basically, I introduced scalability to the stack. Sweet malicious compliance. We’ll see what happens next, when everyone is actually empowered to use AI like the execs want. My prediction is an about-face.

        Edit: sometimes the best way to win a fight is to show the referee that the rules don’t make any fucking sense. Sometimes the referee doesn’t speak English, though, so you have to show them via USD speak. Sometimes they have a hard time hearing too, so you have to let them taste the poison a little. Sometimes they get drunk off that little bit, so you then gotta throw them in the fucking pool and tell them to start swimming.

      • funkless_eck@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        33
        ·
        9 hours ago

        yep. everyone at mine was being praised for creating an agent that turned meetings into JSON and then the JSON into Asana tasks and the Asana tasks into a report and the report into an internal and external email and the email into a slack message and the slack messages and emails into weekly summary.

        Burning thousands of credits for what could be replaced by…

        listening

        • BackgrndNoize@lemmy.world
          link
          fedilink
          English
          arrow-up
          4
          ·
          3 hours ago

          All that and you know for a fact no one’s reading any of that crap. I hate that working as an adult involves working with spineless losers who would rather participate in this clown show than call it out, because being real gets you punished

        • BassTurd@lemmy.world
          link
          fedilink
          English
          arrow-up
          18
          ·
          8 hours ago

          I created a PoC to have an event parse an employees emails, summarize them, then check the calendar and recommend meetings and follow up based on context. It ended up working okay, but it was such a waste of time. This was a C Suite employee that requested, who gets a high volume of junk email. Why would you want AI to (initially requested) auto create meetings for you? That sounds like my nightmare. In the end, it never hit prod thankfully, but, the dev work to get to where I did was awful. Developing AI agents is like guessing and checking until you get close enough. Debugging is brutal and the work is extremely uninspiring.

      • BassTurd@lemmy.world
        link
        fedilink
        English
        arrow-up
        18
        ·
        8 hours ago

        I was put in a position to start building and deploying copilot agents in our company. I discovered in the first day how unprepared our environment was for anything copilot. The default state when you get licenses and an environment to work in, is the wild west. It’s really bad and as is tradition, half cocked and rushed to market. I wrote out pages of notes of things we had to do as a company before even the most simple agents are created for security and governance. I never got the support to implement any changes, so I drug my feet as much as I could on anything I could. For 6 months I successfully never deployed any AI stuff and got out to do full stack dev instead. I created PoC agents, but with hard caveats that none of it was usable in the current state in prod.

        Now prices are increasing and our drive to force agentic has softened a little company wide. I like to think that my semi malicious yet justified slow walking saved us a whole shit load of headache and expenses over the next couple of months as the new copilot pricing hits on June 1.

      • Squizzy@lemmy.world
        link
        fedilink
        English
        arrow-up
        32
        ·
        9 hours ago

        Christ I am only realising now they probably see me asking copilot “why are you so shit” or “Just fuckoff, I’ll do it myself”. They pay for that.

        • jballs@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          6
          ·
          6 hours ago

          Unfortunately that reality is going to quickly turn to “we have to cut staff and freeze salaries this year due to AI spending”.

      • BarneyPiccolo@lemmings.world
        link
        fedilink
        English
        arrow-up
        16
        ·
        10 hours ago

        Now that they’ve poured zillions into it, they are starting to realize that they’ll never get their return on that investment, so now they have to manage the draw down so it doesn’t all crash.

        At least some are recognizing it.