Recent Comments

    LLMs tagged posts

    AI Study reveals Dramatic Reasoning Breakdown in Large Language Models

    July 25, 2024, Categories  Uncategorized

    AI study reveals dramatic reasoning breakdown in LLMs
    Strong fluctuations across AIW problem variations. Also for higher performers, eg GPT-4o, GPT-4 and Claude Opus 3, correct response rates vary strongly from close to 1 to close to 0, despite only slight changes introduced in AIW variations (a color per each variation 1–4). This clearly shows lack of model robustness, hinting basic reasoning deficits. Credit: arXiv (2024). DOI: 10.48550/arxiv.2406.02061

    Even the best AI large language models (LLMs) fail dramatically when it comes to simple logical questions. This is the conclusion of researchers from the Jülich Supercomputing Center (JSC), the School of Electrical and Electronic Engineering at the University of Bristol and the LAION AI laboratory.

    In their paper posted to the arXiv preprint server, titled “Alice in Wonderland: S...

    Read More

    Researchers develop AI-driven Machine-Checking Method for Verifying Software Code

    January 5, 2024, Categories  Physics, Technology/Electronics

    software code
    Credit: Pixabay/CC0 Public Domain

    A team of computer scientists led by the University of Massachusetts Amherst recently announced a new method for automatically generating whole proofs that can be used to prevent software bugs and verify that the underlying code is correct.

    This new method, called Baldur, leverages the artificial intelligence power of large language models (LLMs), and when combined with the state-of-the-art tool Thor, yields unprecedented efficacy of nearly 66%. The team was recently awarded a Distinguished Paper award at the ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering.

    “We have unfortunately come to expect that our software is buggy, despite the fact that it is everywhere and we all use it every day...

    Read More