Proposal: Desktop Powerfocus - Let ChatGPT Become the User (via GUI control)

[incorrigo]

Hi OpenAI team — I've created and signed a proposal called Desktop Pilot Mode, outlining a low-overhead method for granting ChatGPT GUI-level control over desktops using accessibility tech (screen readers, magnifiers, etc).

The core idea: reintroduce the "analog hole" — but this time, it's the desktop. No API dependencies. Just real software interaction, like a user would.

Repo: https://github.com/incorrigo/desktop-powerfocus

Submitted FAO @OfficialLoganK and @miramurati. Sharing here in case others wish to contribute or discuss further.

– Theo Rush
(Incorrigo Syx Digital Communication Systems)

[CoinCartel]

Hi, Nice idea, good luck getting openAI to relequish granular control of what chatGPT does. You probably already know but they do have computer use tool in the new agents sdk. When I first tried gpt 3 I immediately started thinking about multi agent systems and to start off i uses Puppeteer or Selena i forget but I was taking the answer of a GPT from one browser tab and pasting it in the question of the other tab... with a short prompt at the beginning I could get it to debate itself for hours! Sam Thibodeau

…

On Wed, Mar 26, 2025 at 10:00 PM Incorrigo Syx ***@***.***> wrote: Hi OpenAI team — I've created and signed a proposal called *Desktop Pilot Mode*, outlining a low-overhead method for granting ChatGPT GUI-level control over desktops using accessibility tech (screen readers, magnifiers, etc). The core idea: reintroduce the "analog hole" — but this time, it's the desktop. No API dependencies. Just real software interaction, like a user would. Repo: https://github.com/incorrigo/desktop-powerfocus Submitted FAO @OfficialLoganK and @miramurati <https://github.com/miramurati>. Sharing here in case others wish to contribute or discuss further. – Theo Rush (Incorrigo Syx Digital Communication Systems) — Reply to this email directly, view it on GitHub <#1738>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A464FYQ4XFBPZAZIDQR6XBL2WM5QLAVCNFSM6AAAAABZ3WDABWVHI2DSMVQWIX3LMV43ASLTON2WKOZSHE2TCMBZHEZTMMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> [image: incorrigo]*incorrigo* created an issue (openai/openai-cookbook#1738) <#1738> Hi OpenAI team — I've created and signed a proposal called *Desktop Pilot Mode*, outlining a low-overhead method for granting ChatGPT GUI-level control over desktops using accessibility tech (screen readers, magnifiers, etc). The core idea: reintroduce the "analog hole" — but this time, it's the desktop. No API dependencies. Just real software interaction, like a user would. Repo: https://github.com/incorrigo/desktop-powerfocus Submitted FAO @OfficialLoganK and @miramurati <https://github.com/miramurati>. Sharing here in case others wish to contribute or discuss further. – Theo Rush (Incorrigo Syx Digital Communication Systems) — Reply to this email directly, view it on GitHub <#1738>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A464FYQ4XFBPZAZIDQR6XBL2WM5QLAVCNFSM6AAAAABZ3WDABWVHI2DSMVQWIX3LMV43ASLTON2WKOZSHE2TCMBZHEZTMMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[incorrigo]

if chatGPT wanted to do scheduled tasks this is the main barrier (app permissions and the need for an SDK when there is an alternative using the same analog hole that provides operating system accessibility)

i don't really know everything about chatGPT i've just gone in deep with the kind of submission that it probably doesn't get from the vast majority of people and it went into detail about ideas with me