p3x.debrianpeiris, brianpeiris@lemmy.ca
Instance: lemmy.ca
Joined: 3 years ago
Posts: 31
Comments: 87
Posts and Comments by brianpeiris, brianpeiris@lemmy.ca
c/technology
c/fuck_ai
c/linux
Comments by brianpeiris, brianpeiris@lemmy.ca
Not The OnionThey started the meeting with a prayer about “keeping their minds on the children”, followed by the most robotic sounding pledge of allegiance I’ve ever heard, and then they proceeded to pardon the predator, mostly using religious “grace” as a justification. America!
Programmer Humor@programming.dev
Buddhist Copilot builds apps with sublime coding standards, and on the last iteration it runs rm -rf * .git before it recites a koan on impermanence.
I suspect the problem is that there are many developers nowadays who don’t care about code quality, actual engineering, and maintenance. So the people who are complaining are right to be concerned that there is going to be a ton of slop code produced by AI-bro developers, and the developers who actually care will be left to deal with the aftermath. I’d be very happy if lead developers are prepared to try things with AI, and importantly to throw the output away if it doesn’t meet coding standards. Instead I think even lead developers and CTOs are chasing “productivity” metrics, which just translates to a ton of sloppy code.
Why is this so downvoted?
Here you go. It’s very short:
In their recent Comment article, Eddy Keming Chen et al. argue that current large language models (LLMs) already display human-level intelligence, based on behavioural evidence (see Nature 650, 36–40; 2026). I suggest that this framing obscures a fundamental asymmetry.
The authors treat human minds and LLMs as two comparable systems: effectively, two black boxes that are evaluated by their outputs. But this symmetry is fictitious. Human intelligence is a natural phenomenon, from which the very concept of intelligence is reconstructed. The generative mechanisms of the human mind are not yet fully understood. By contrast, LLMs are systems that are designed and built. Their operating principles — statistical optimization of token prediction — are known, even if internal complexity makes it difficult to retrace the steps that produce the outputs. LLMs are complex, but they are not inherently mysterious black boxes.
When we attribute intelligence to humans, no alternative explanation for their cognitive behaviour is available, nor is it needed. But there is a sufficient explanation for the behaviour of LLMs, which does not infer understanding or intelligence: the known generative mechanism itself.
This does not mean that artificial general intelligence is impossible in principle. But establishing it would require evidence that the cognitive behaviour of a system cannot be fully accounted for by its known generative mechanism alone.
Science Memes@mander.xyz
Here’s the vid: https://www.youtube.com/shorts/GAMkRJdu9j4
I don’t quite understand the concerns in the comments here. The original blog post seems reasonable: https://www.waterfox.com/blog/15-years-of-forking/#what-waterfox-is-in-2026
FuturologyFinally got around to watching this. Thanks for sharing.
Yes, the LLMs received credit for each level even if they didn’t complete the entire environment.
They have some replays of tasks on their website: https://arcprize.org/tasks
Here’s one where the human completed all 9 levels in 1458 actions, but the LLM completed only one level in 24 actions, then struggled for 190 actions until it timed-out, I guess. The LLM scored 2.8% because of the weighted average, I think. I didn’t take the time to all do the math, and I’m not sure if the replay action count is accurate, but it gives you an idea.
Human: https://arcprize.org/replay/0d461c1c-21e5-4dc8-b263-9922332a6485
LLM: https://arcprize.org/replay/cc821983-3975-4ae4-a70b-e031f6807bb0
I like the trend of refining existing tools. You take tried-and-true commands and shave off the rough edges and quirks.
I use ripgrep instead of grep, fd instead of find, scm_breeze on top of git, dust instead of du, duf instead of df, z over cd, and xh instead of curl
You can really only judge fairness of the score if you understand the scoring criteria. It is a relative score where the baseline is 100% for humans – i.e. A task was only included in the challenge if at least two people in the panel of humans were able to solve it completely, and their action count is a measure of efficiency. This is the baseline used as a point of comparison.
From the Technical Report:
The procedure can be summarized as follows:
• “Score the AI test taker by its per-level action efficiency” - For each level that the test taker completes, count the number of actions that it took.
• “As compared to human baseline” - For each level that is counted, compare the AI agent’s action count to a human baseline, which we define as the second-best human action count. Ex: If the second-best human completed a level in only 10 actions, but the AI agent took 100 to complete it, then the AI agent scores (10/100)^2 for that level, which gets reported as 1%. Note that level scoring is calculated using the square of efficiency.
• “Normalized per environment” - Each level is scored in isolation. Each individual level will get a score between 0% (very inefficient) 100% (matches or surpasses human level efficiency). The environment score will be a weighted-average of level score across all levels of that environment.
• “Across all environments” - The total score will be the sum of individual environment scores divided
by the total number of environments. This will be a score between 0% and 100%.
So the humans “scored 100%” because that is the baseline by definition, and the AIs are evaluated at how close they got to human correctness and efficiency. So a score of 0.26% is 1/0.0026 ~= 385 times less efficient (and correct) compared to humans.
The goal of the ARC organization is to continually measure progress towards AGI, not come up with some predictive threshold for when AGI is achieved.
As long as they can continue to measure a gap between “easy for humans” and “hard for AI”, they will continue releasing new iterations of this ARC-AGI challenge series. Currently they do that about once a year.
More detail about the mission here: https://arcprize.org/arc-agi
It’s true that frontier models got better at the previous challenges, but it’s worth noting that they’re still not quite at human level even with those simpler tasks.
Also, each generation of the challenge tries to close loopholes that newer models would exploit, like brute-forcing the training with tons of synthesized tasks and solutions, over-fitting to these particular kinds of tasks, and issues with the similarities between the tasks in the challenge.
A common strategy in past challenges was to generate thousands of similar tasks, and you can imagine the big AI companies were able to do that at massive scale for their frontier models.
There’s a column linking to replays in the table of tasks here: https://arcprize.org/tasks
This is my rough upper-bound estimate based on the Technical Report. Human participants were paid to complete and evaluate the tasks at an average fixed fee of $128 plus $5 for solved tasks. So if a panel of humans were tasked with solving the 25 tasks in the public test set, it would be an average of $250 per person. Although, looking at it again, the costs listed for the LLMs is per task, so it would actually be more like $10 per human per task. In any case it’s one or two orders of magnitude less than the LLMs.
Participants received a fixed participation fee of $115–$140 for completing the session, along with a $5
performance-based incentive for each environment successfully solved
ARC-AGI-3 Launch event - Shared publicly live on March 25 in San Francisco at Y Combinator HQ, featuring a fireside conversation between François Chollet (creator, ARC-AGI) and Sam Altman (CEO, OpenAI) on measuring intelligence on the path to AGI.
François Chollet is a software engineer, artificial intelligence researcher, and former Senior Staff Engineer at Google. Chollet is the creator of the Keras deep-learning library released in 2015.
Here is the full text of the suit:
https://www.courthousenews.com/wp-content/uploads/2026/03/tumbler-ridge-openAI.pdf
They are suing under 4 acts:
- The Negligence Act, RSBC 1996, c. 333
- The Court Order Interest Act, RSBC 1996, c. 79
- Court Jurisdiction and Proceedings Transfer Act, RSBC 2003, c. 28
- Health Care Costs Recovery Act. SBC 2008, c. 27
My interpretation of the FCA is that if a family member dies, you are able to sue for damages. It does not say anything about pain, suffering or loss of companionship.
The second link is about corporate benefits claims in the event of a relative’s death, not about damages.
The family suing still has their daughter alive in hospital. So neither applies anyway.
Am I reading it wrong? It seems the whole point of the Family Compensation Act is to allow families to seek compensation if a family member has died. Though in this case, the family that is suing still has their daughter alive in hospital. Anyway, I don’t think a lawyer would have taken the case on if it would have been rejected.
If anything, I suspect OpenAI will be able to get away with a sizable private settlement, given their enormous funding, and the detailed chat logs will never be public.
They started the meeting with a prayer about “keeping their minds on the children”, followed by the most robotic sounding pledge of allegiance I’ve ever heard, and then they proceeded to pardon the predator, mostly using religious “grace” as a justification. America!
Buddhist Copilot builds apps with sublime coding standards, and on the last iteration it runs
rm -rf * .gitbefore it recites a koan on impermanence.I suspect the problem is that there are many developers nowadays who don’t care about code quality, actual engineering, and maintenance. So the people who are complaining are right to be concerned that there is going to be a ton of slop code produced by AI-bro developers, and the developers who actually care will be left to deal with the aftermath. I’d be very happy if lead developers are prepared to try things with AI, and importantly to throw the output away if it doesn’t meet coding standards. Instead I think even lead developers and CTOs are chasing “productivity” metrics, which just translates to a ton of sloppy code.
Why is this so downvoted?
Here you go. It’s very short:
Here’s the vid: https://www.youtube.com/shorts/GAMkRJdu9j4
I don’t quite understand the concerns in the comments here. The original blog post seems reasonable: https://www.waterfox.com/blog/15-years-of-forking/#what-waterfox-is-in-2026
This is AI generated, isn’t it?
Finally got around to watching this. Thanks for sharing.
Yes, the LLMs received credit for each level even if they didn’t complete the entire environment.
They have some replays of tasks on their website: https://arcprize.org/tasks
Here’s one where the human completed all 9 levels in 1458 actions, but the LLM completed only one level in 24 actions, then struggled for 190 actions until it timed-out, I guess. The LLM scored 2.8% because of the weighted average, I think. I didn’t take the time to all do the math, and I’m not sure if the replay action count is accurate, but it gives you an idea.
Human: https://arcprize.org/replay/0d461c1c-21e5-4dc8-b263-9922332a6485
LLM: https://arcprize.org/replay/cc821983-3975-4ae4-a70b-e031f6807bb0
I like the trend of refining existing tools. You take tried-and-true commands and shave off the rough edges and quirks. I use
ripgrepinstead ofgrep,fdinstead offind,scm_breezeon top ofgit,dustinstead ofdu,dufinstead ofdf,zovercd, andxhinstead ofcurlYou can really only judge fairness of the score if you understand the scoring criteria. It is a relative score where the baseline is 100% for humans – i.e. A task was only included in the challenge if at least two people in the panel of humans were able to solve it completely, and their action count is a measure of efficiency. This is the baseline used as a point of comparison.
From the Technical Report:
by the total number of environments. This will be a score between 0% and 100%.
So the humans “scored 100%” because that is the baseline by definition, and the AIs are evaluated at how close they got to human correctness and efficiency. So a score of 0.26% is 1/0.0026 ~= 385 times less efficient (and correct) compared to humans.
The goal of the ARC organization is to continually measure progress towards AGI, not come up with some predictive threshold for when AGI is achieved.
As long as they can continue to measure a gap between “easy for humans” and “hard for AI”, they will continue releasing new iterations of this ARC-AGI challenge series. Currently they do that about once a year.
More detail about the mission here: https://arcprize.org/arc-agi
It’s true that frontier models got better at the previous challenges, but it’s worth noting that they’re still not quite at human level even with those simpler tasks.
Also, each generation of the challenge tries to close loopholes that newer models would exploit, like brute-forcing the training with tons of synthesized tasks and solutions, over-fitting to these particular kinds of tasks, and issues with the similarities between the tasks in the challenge.
A common strategy in past challenges was to generate thousands of similar tasks, and you can imagine the big AI companies were able to do that at massive scale for their frontier models.
There’s a column linking to replays in the table of tasks here: https://arcprize.org/tasks
This is my rough upper-bound estimate based on the Technical Report. Human participants were paid to complete and evaluate the tasks at an average fixed fee of $128 plus $5 for solved tasks. So if a panel of humans were tasked with solving the 25 tasks in the public test set, it would be an average of $250 per person. Although, looking at it again, the costs listed for the LLMs is per task, so it would actually be more like $10 per human per task. In any case it’s one or two orders of magnitude less than the LLMs.
performance-based incentive for each environment successfully solved
https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf
Announcing ARC-AGI-3 - A benchmark that tests if AI can explore, learn, and adapt in unfamiliar situations. Humans score 100%. Frontier AI scores 0.26%. (arcprize.org)
The ARC Prize organization designs benchmarks which are specifically crafted to demonstrate tasks that humans complete easily, but are difficult for AIs like LLMs, “Reasoning” models, and Agentic frameworks.
Not By AI - A Badge for Human-Created Content (notbyai.fyi)
Here is the full text of the suit:
https://www.courthousenews.com/wp-content/uploads/2026/03/tumbler-ridge-openAI.pdf
They are suing under 4 acts: