▲ 500 ▼ VCs are starting to partner with private equity to buy up call centers, accounting firms and other "mature companies" to replace their operations with AI (techcrunch.com) submitted 1 year ago by dantheclamman@lemmy.world to c/technology@lemmy.world 148 comments fedilink hide all child comments
[–] cyrano@lemmy.dbzer0.com 47 points 1 year ago (4 children) lol accounting…. permalink fedilink source hideshow 8 child comments replies: [+] Initiateofthevoid@lemmy.dbzer0.com 22 points 1 year ago* (last edited 1 year ago) (2 children) [deleted] permalink fedilink source parent hideshow 4 child comments replies: [+] futatorius@lemm.ee 7 points 1 year ago* (last edited 1 year ago) [deleted] permalink fedilink source parent [–] vivendi@programming.dev 5 points 1 year ago (2 children) For usage like that you'd wire an LLM into a tool use workflow with whatever accounting software you have. The LLM would make queries to the rigid, non-hallucinating accounting system. I still don't think it would be anywhere close to a good idea because you'd need a lot of safeguards and also fuck your accounting and you'll have some unpleasant meetings with the local equivalent of the IRS. permalink fedilink source parent hideshow 4 child comments replies: [+] futatorius@lemm.ee 4 points 1 year ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 1 point 1 year ago But ERP is not a cool buzzword, hence it can fuck off we're in 2025 permalink fedilink source parent [–] pinball_wizard@lemmy.zip 4 points 1 year ago (1 child) The LLM would make queries to the rigid, non-hallucinating accounting system. And then sometimes adds a halucination before returning an answer - particularly when it encournters anything it wasn't trained on, like important moments when business leaders should be taking a closer look. There's not enough popcorn in the world for the shitshow that is coming. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 0 points 1 year ago (1 child) You're misunderstanding tool use, the LLM only queries something to be done then the actual system returns the result. You can also summarize the result or something but hallucinations in that workload are remarkably low (however without tuning they can drop important information from the response) The place where it can hallucinate is generating steps for your natural language query, or the entry stage. That's why you need to safeguard like your ass depends on it. (Which it does, if your boss is stupid enough) permalink fedilink source parent hideshow 2 child comments replies: [–] pinball_wizard@lemmy.zip 1 point 1 year ago* (last edited 1 year ago) I'm quite aware that it's less likely to technically hallucinate in these cases. But focusing on that technicality doesn't serve users well. These (interesting and useful) use cases do not address the core issue that the query was written by the LLM, without expert oversight, which still leads to situations that are effectively halucinations. Technically, it is returning a "correct" direct answer to a question that no rational actor would ever have asked. But when a halucinated (correct looking but deeply flawed) query is sent to the system of record, it's most honest to still call the results a halucination, as well. Even though they are technically real data, just astonishingly poorly chosen real data. The meaningless, correct-looking and wrong result for the end user is still just going to be called a halucination, by common folks. For common usage, it's important not to promise end users that these scenarios are free of halucination. You and I understand that technically, they're not getting back a halucination, just an answer to a bad question. But for the end user to understand how to use the tool safely, they still need to know that a meaningless correct looking and wrong answer is still possible (and today, still also likely). permalink fedilink source parent [–] Korhaka@sopuli.xyz 10 points 1 year ago (2 children) How easy will it be to fool the AI into getting the company in legal trouble? Oh well. permalink fedilink source parent hideshow 4 child comments replies: [–] pinball_wizard@lemmy.zip 3 points 1 year ago Some would call it effortless, even. permalink fedilink source parent [–] Meron35@lemmy.world 1 point 1 year ago NYC's AI chatbot was caught telling businesses to break the law. The city isn't taking it down | AP News - https://apnews.com/article/new-york-city-chatbot-misinformation-6ebc71db5b770b9969c906a7ee4fae21 permalink fedilink source parent [–] Melvin_Ferd@lemmy.world 4 points 1 year ago Hey boss. Think they're using chatgpt for that? permalink fedilink source parent [–] vivendi@programming.dev 0 points 1 year ago (1 child) This is because auto regressive LLMs work on high level "Tokens". There are LLM experiments which can access byte information, to correctly answer such questions. Also, they don't want to support you omegalul do you really think call centers are hired to give a fuck about you? this is intentional permalink fedilink source parent hideshow 2 child comments replies: [–] Repelle@lemmy.world 5 points 1 year ago (1 child) I don’t think that’s the full explanation though, because there are examples of models that will correctly spell out the word first (ie, it knows the component letter tokens) and still miscount the letters after doing so. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 0 points 1 year ago (1 child) No, this literally is the explanation. The model understands the concept of "Strawberry", It can output from the model (and that itself is very complicated) in English as Strawberry, jn Persian as توت فرنگی and so on. But the model does not understand how many Rs exist in Strawberry or how many ت exist in توت فرنگی permalink fedilink source parent hideshow 2 child comments replies: [–] Repelle@lemmy.world 3 points 1 year ago* (1 child) I’m talking about models printing out the component letters first not just printing out the full word. As in “S - T - R - A - W - B - E - R - R - Y” then getting the answer wrong. You’re absolutely right that it reads in words at a time encoded to vectors, but if it’s holding a relationship from that coding to the component spelling, which it seems it must be given it is outputting the letters individually, then something else is wrong. I’m not saying all models fail this way, and I’m sure many fail in exactly the way you describe, but I have seen this failure mode (which is what I was trying to describe) and in that case an alternate explanation would be necessary. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 5 points 1 year ago* The model ISN'T outputing the letters individually, binary models (as I mentioned) do; not transformers. The model output is more like Strawberry Tokens can be a letter, part of a word, any single lexeme, any word, or even multiple words ("let be") Okay I did a shit job demonstrating the time axis. The model doesn't know the underlying letters of the previous tokens and this processes is going forward in time permalink fedilink source parent
[+] Initiateofthevoid@lemmy.dbzer0.com 22 points 1 year ago* (last edited 1 year ago) (2 children) [deleted] permalink fedilink source parent hideshow 4 child comments replies: [+] futatorius@lemm.ee 7 points 1 year ago* (last edited 1 year ago) [deleted] permalink fedilink source parent [–] vivendi@programming.dev 5 points 1 year ago (2 children) For usage like that you'd wire an LLM into a tool use workflow with whatever accounting software you have. The LLM would make queries to the rigid, non-hallucinating accounting system. I still don't think it would be anywhere close to a good idea because you'd need a lot of safeguards and also fuck your accounting and you'll have some unpleasant meetings with the local equivalent of the IRS. permalink fedilink source parent hideshow 4 child comments replies: [+] futatorius@lemm.ee 4 points 1 year ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 1 point 1 year ago But ERP is not a cool buzzword, hence it can fuck off we're in 2025 permalink fedilink source parent [–] pinball_wizard@lemmy.zip 4 points 1 year ago (1 child) The LLM would make queries to the rigid, non-hallucinating accounting system. And then sometimes adds a halucination before returning an answer - particularly when it encournters anything it wasn't trained on, like important moments when business leaders should be taking a closer look. There's not enough popcorn in the world for the shitshow that is coming. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 0 points 1 year ago (1 child) You're misunderstanding tool use, the LLM only queries something to be done then the actual system returns the result. You can also summarize the result or something but hallucinations in that workload are remarkably low (however without tuning they can drop important information from the response) The place where it can hallucinate is generating steps for your natural language query, or the entry stage. That's why you need to safeguard like your ass depends on it. (Which it does, if your boss is stupid enough) permalink fedilink source parent hideshow 2 child comments replies: [–] pinball_wizard@lemmy.zip 1 point 1 year ago* (last edited 1 year ago) I'm quite aware that it's less likely to technically hallucinate in these cases. But focusing on that technicality doesn't serve users well. These (interesting and useful) use cases do not address the core issue that the query was written by the LLM, without expert oversight, which still leads to situations that are effectively halucinations. Technically, it is returning a "correct" direct answer to a question that no rational actor would ever have asked. But when a halucinated (correct looking but deeply flawed) query is sent to the system of record, it's most honest to still call the results a halucination, as well. Even though they are technically real data, just astonishingly poorly chosen real data. The meaningless, correct-looking and wrong result for the end user is still just going to be called a halucination, by common folks. For common usage, it's important not to promise end users that these scenarios are free of halucination. You and I understand that technically, they're not getting back a halucination, just an answer to a bad question. But for the end user to understand how to use the tool safely, they still need to know that a meaningless correct looking and wrong answer is still possible (and today, still also likely). permalink fedilink source parent
[+] futatorius@lemm.ee 7 points 1 year ago* (last edited 1 year ago) [deleted] permalink fedilink source parent
[–] vivendi@programming.dev 5 points 1 year ago (2 children) For usage like that you'd wire an LLM into a tool use workflow with whatever accounting software you have. The LLM would make queries to the rigid, non-hallucinating accounting system. I still don't think it would be anywhere close to a good idea because you'd need a lot of safeguards and also fuck your accounting and you'll have some unpleasant meetings with the local equivalent of the IRS. permalink fedilink source parent hideshow 4 child comments replies: [+] futatorius@lemm.ee 4 points 1 year ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 1 point 1 year ago But ERP is not a cool buzzword, hence it can fuck off we're in 2025 permalink fedilink source parent [–] pinball_wizard@lemmy.zip 4 points 1 year ago (1 child) The LLM would make queries to the rigid, non-hallucinating accounting system. And then sometimes adds a halucination before returning an answer - particularly when it encournters anything it wasn't trained on, like important moments when business leaders should be taking a closer look. There's not enough popcorn in the world for the shitshow that is coming. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 0 points 1 year ago (1 child) You're misunderstanding tool use, the LLM only queries something to be done then the actual system returns the result. You can also summarize the result or something but hallucinations in that workload are remarkably low (however without tuning they can drop important information from the response) The place where it can hallucinate is generating steps for your natural language query, or the entry stage. That's why you need to safeguard like your ass depends on it. (Which it does, if your boss is stupid enough) permalink fedilink source parent hideshow 2 child comments replies: [–] pinball_wizard@lemmy.zip 1 point 1 year ago* (last edited 1 year ago) I'm quite aware that it's less likely to technically hallucinate in these cases. But focusing on that technicality doesn't serve users well. These (interesting and useful) use cases do not address the core issue that the query was written by the LLM, without expert oversight, which still leads to situations that are effectively halucinations. Technically, it is returning a "correct" direct answer to a question that no rational actor would ever have asked. But when a halucinated (correct looking but deeply flawed) query is sent to the system of record, it's most honest to still call the results a halucination, as well. Even though they are technically real data, just astonishingly poorly chosen real data. The meaningless, correct-looking and wrong result for the end user is still just going to be called a halucination, by common folks. For common usage, it's important not to promise end users that these scenarios are free of halucination. You and I understand that technically, they're not getting back a halucination, just an answer to a bad question. But for the end user to understand how to use the tool safely, they still need to know that a meaningless correct looking and wrong answer is still possible (and today, still also likely). permalink fedilink source parent
[+] futatorius@lemm.ee 4 points 1 year ago* (last edited 1 year ago) (1 child) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 1 point 1 year ago But ERP is not a cool buzzword, hence it can fuck off we're in 2025 permalink fedilink source parent
[–] vivendi@programming.dev 1 point 1 year ago But ERP is not a cool buzzword, hence it can fuck off we're in 2025 permalink fedilink source parent
[–] pinball_wizard@lemmy.zip 4 points 1 year ago (1 child) The LLM would make queries to the rigid, non-hallucinating accounting system. And then sometimes adds a halucination before returning an answer - particularly when it encournters anything it wasn't trained on, like important moments when business leaders should be taking a closer look. There's not enough popcorn in the world for the shitshow that is coming. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 0 points 1 year ago (1 child) You're misunderstanding tool use, the LLM only queries something to be done then the actual system returns the result. You can also summarize the result or something but hallucinations in that workload are remarkably low (however without tuning they can drop important information from the response) The place where it can hallucinate is generating steps for your natural language query, or the entry stage. That's why you need to safeguard like your ass depends on it. (Which it does, if your boss is stupid enough) permalink fedilink source parent hideshow 2 child comments replies: [–] pinball_wizard@lemmy.zip 1 point 1 year ago* (last edited 1 year ago) I'm quite aware that it's less likely to technically hallucinate in these cases. But focusing on that technicality doesn't serve users well. These (interesting and useful) use cases do not address the core issue that the query was written by the LLM, without expert oversight, which still leads to situations that are effectively halucinations. Technically, it is returning a "correct" direct answer to a question that no rational actor would ever have asked. But when a halucinated (correct looking but deeply flawed) query is sent to the system of record, it's most honest to still call the results a halucination, as well. Even though they are technically real data, just astonishingly poorly chosen real data. The meaningless, correct-looking and wrong result for the end user is still just going to be called a halucination, by common folks. For common usage, it's important not to promise end users that these scenarios are free of halucination. You and I understand that technically, they're not getting back a halucination, just an answer to a bad question. But for the end user to understand how to use the tool safely, they still need to know that a meaningless correct looking and wrong answer is still possible (and today, still also likely). permalink fedilink source parent
[–] vivendi@programming.dev 0 points 1 year ago (1 child) You're misunderstanding tool use, the LLM only queries something to be done then the actual system returns the result. You can also summarize the result or something but hallucinations in that workload are remarkably low (however without tuning they can drop important information from the response) The place where it can hallucinate is generating steps for your natural language query, or the entry stage. That's why you need to safeguard like your ass depends on it. (Which it does, if your boss is stupid enough) permalink fedilink source parent hideshow 2 child comments replies: [–] pinball_wizard@lemmy.zip 1 point 1 year ago* (last edited 1 year ago) I'm quite aware that it's less likely to technically hallucinate in these cases. But focusing on that technicality doesn't serve users well. These (interesting and useful) use cases do not address the core issue that the query was written by the LLM, without expert oversight, which still leads to situations that are effectively halucinations. Technically, it is returning a "correct" direct answer to a question that no rational actor would ever have asked. But when a halucinated (correct looking but deeply flawed) query is sent to the system of record, it's most honest to still call the results a halucination, as well. Even though they are technically real data, just astonishingly poorly chosen real data. The meaningless, correct-looking and wrong result for the end user is still just going to be called a halucination, by common folks. For common usage, it's important not to promise end users that these scenarios are free of halucination. You and I understand that technically, they're not getting back a halucination, just an answer to a bad question. But for the end user to understand how to use the tool safely, they still need to know that a meaningless correct looking and wrong answer is still possible (and today, still also likely). permalink fedilink source parent
[–] pinball_wizard@lemmy.zip 1 point 1 year ago* (last edited 1 year ago) I'm quite aware that it's less likely to technically hallucinate in these cases. But focusing on that technicality doesn't serve users well. These (interesting and useful) use cases do not address the core issue that the query was written by the LLM, without expert oversight, which still leads to situations that are effectively halucinations. Technically, it is returning a "correct" direct answer to a question that no rational actor would ever have asked. But when a halucinated (correct looking but deeply flawed) query is sent to the system of record, it's most honest to still call the results a halucination, as well. Even though they are technically real data, just astonishingly poorly chosen real data. The meaningless, correct-looking and wrong result for the end user is still just going to be called a halucination, by common folks. For common usage, it's important not to promise end users that these scenarios are free of halucination. You and I understand that technically, they're not getting back a halucination, just an answer to a bad question. But for the end user to understand how to use the tool safely, they still need to know that a meaningless correct looking and wrong answer is still possible (and today, still also likely). permalink fedilink source parent
[–] Korhaka@sopuli.xyz 10 points 1 year ago (2 children) How easy will it be to fool the AI into getting the company in legal trouble? Oh well. permalink fedilink source parent hideshow 4 child comments replies: [–] pinball_wizard@lemmy.zip 3 points 1 year ago Some would call it effortless, even. permalink fedilink source parent [–] Meron35@lemmy.world 1 point 1 year ago NYC's AI chatbot was caught telling businesses to break the law. The city isn't taking it down | AP News - https://apnews.com/article/new-york-city-chatbot-misinformation-6ebc71db5b770b9969c906a7ee4fae21 permalink fedilink source parent
[–] pinball_wizard@lemmy.zip 3 points 1 year ago Some would call it effortless, even. permalink fedilink source parent
[–] Meron35@lemmy.world 1 point 1 year ago NYC's AI chatbot was caught telling businesses to break the law. The city isn't taking it down | AP News - https://apnews.com/article/new-york-city-chatbot-misinformation-6ebc71db5b770b9969c906a7ee4fae21 permalink fedilink source parent
[–] Melvin_Ferd@lemmy.world 4 points 1 year ago Hey boss. Think they're using chatgpt for that? permalink fedilink source parent
[–] vivendi@programming.dev 0 points 1 year ago (1 child) This is because auto regressive LLMs work on high level "Tokens". There are LLM experiments which can access byte information, to correctly answer such questions. Also, they don't want to support you omegalul do you really think call centers are hired to give a fuck about you? this is intentional permalink fedilink source parent hideshow 2 child comments replies: [–] Repelle@lemmy.world 5 points 1 year ago (1 child) I don’t think that’s the full explanation though, because there are examples of models that will correctly spell out the word first (ie, it knows the component letter tokens) and still miscount the letters after doing so. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 0 points 1 year ago (1 child) No, this literally is the explanation. The model understands the concept of "Strawberry", It can output from the model (and that itself is very complicated) in English as Strawberry, jn Persian as توت فرنگی and so on. But the model does not understand how many Rs exist in Strawberry or how many ت exist in توت فرنگی permalink fedilink source parent hideshow 2 child comments replies: [–] Repelle@lemmy.world 3 points 1 year ago* (1 child) I’m talking about models printing out the component letters first not just printing out the full word. As in “S - T - R - A - W - B - E - R - R - Y” then getting the answer wrong. You’re absolutely right that it reads in words at a time encoded to vectors, but if it’s holding a relationship from that coding to the component spelling, which it seems it must be given it is outputting the letters individually, then something else is wrong. I’m not saying all models fail this way, and I’m sure many fail in exactly the way you describe, but I have seen this failure mode (which is what I was trying to describe) and in that case an alternate explanation would be necessary. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 5 points 1 year ago* The model ISN'T outputing the letters individually, binary models (as I mentioned) do; not transformers. The model output is more like Strawberry Tokens can be a letter, part of a word, any single lexeme, any word, or even multiple words ("let be") Okay I did a shit job demonstrating the time axis. The model doesn't know the underlying letters of the previous tokens and this processes is going forward in time permalink fedilink source parent
[–] Repelle@lemmy.world 5 points 1 year ago (1 child) I don’t think that’s the full explanation though, because there are examples of models that will correctly spell out the word first (ie, it knows the component letter tokens) and still miscount the letters after doing so. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 0 points 1 year ago (1 child) No, this literally is the explanation. The model understands the concept of "Strawberry", It can output from the model (and that itself is very complicated) in English as Strawberry, jn Persian as توت فرنگی and so on. But the model does not understand how many Rs exist in Strawberry or how many ت exist in توت فرنگی permalink fedilink source parent hideshow 2 child comments replies: [–] Repelle@lemmy.world 3 points 1 year ago* (1 child) I’m talking about models printing out the component letters first not just printing out the full word. As in “S - T - R - A - W - B - E - R - R - Y” then getting the answer wrong. You’re absolutely right that it reads in words at a time encoded to vectors, but if it’s holding a relationship from that coding to the component spelling, which it seems it must be given it is outputting the letters individually, then something else is wrong. I’m not saying all models fail this way, and I’m sure many fail in exactly the way you describe, but I have seen this failure mode (which is what I was trying to describe) and in that case an alternate explanation would be necessary. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 5 points 1 year ago* The model ISN'T outputing the letters individually, binary models (as I mentioned) do; not transformers. The model output is more like Strawberry Tokens can be a letter, part of a word, any single lexeme, any word, or even multiple words ("let be") Okay I did a shit job demonstrating the time axis. The model doesn't know the underlying letters of the previous tokens and this processes is going forward in time permalink fedilink source parent
[–] vivendi@programming.dev 0 points 1 year ago (1 child) No, this literally is the explanation. The model understands the concept of "Strawberry", It can output from the model (and that itself is very complicated) in English as Strawberry, jn Persian as توت فرنگی and so on. But the model does not understand how many Rs exist in Strawberry or how many ت exist in توت فرنگی permalink fedilink source parent hideshow 2 child comments replies: [–] Repelle@lemmy.world 3 points 1 year ago* (1 child) I’m talking about models printing out the component letters first not just printing out the full word. As in “S - T - R - A - W - B - E - R - R - Y” then getting the answer wrong. You’re absolutely right that it reads in words at a time encoded to vectors, but if it’s holding a relationship from that coding to the component spelling, which it seems it must be given it is outputting the letters individually, then something else is wrong. I’m not saying all models fail this way, and I’m sure many fail in exactly the way you describe, but I have seen this failure mode (which is what I was trying to describe) and in that case an alternate explanation would be necessary. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 5 points 1 year ago* The model ISN'T outputing the letters individually, binary models (as I mentioned) do; not transformers. The model output is more like Strawberry Tokens can be a letter, part of a word, any single lexeme, any word, or even multiple words ("let be") Okay I did a shit job demonstrating the time axis. The model doesn't know the underlying letters of the previous tokens and this processes is going forward in time permalink fedilink source parent
[–] Repelle@lemmy.world 3 points 1 year ago* (1 child) I’m talking about models printing out the component letters first not just printing out the full word. As in “S - T - R - A - W - B - E - R - R - Y” then getting the answer wrong. You’re absolutely right that it reads in words at a time encoded to vectors, but if it’s holding a relationship from that coding to the component spelling, which it seems it must be given it is outputting the letters individually, then something else is wrong. I’m not saying all models fail this way, and I’m sure many fail in exactly the way you describe, but I have seen this failure mode (which is what I was trying to describe) and in that case an alternate explanation would be necessary. permalink fedilink source parent hideshow 2 child comments replies: [–] vivendi@programming.dev 5 points 1 year ago* The model ISN'T outputing the letters individually, binary models (as I mentioned) do; not transformers. The model output is more like Strawberry Tokens can be a letter, part of a word, any single lexeme, any word, or even multiple words ("let be") Okay I did a shit job demonstrating the time axis. The model doesn't know the underlying letters of the previous tokens and this processes is going forward in time permalink fedilink source parent
[–] vivendi@programming.dev 5 points 1 year ago* The model ISN'T outputing the letters individually, binary models (as I mentioned) do; not transformers. The model output is more like Strawberry Tokens can be a letter, part of a word, any single lexeme, any word, or even multiple words ("let be") Okay I did a shit job demonstrating the time axis. The model doesn't know the underlying letters of the previous tokens and this processes is going forward in time permalink fedilink source parent