<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Augusta B Lina on Medium]]></title>
        <description><![CDATA[Stories by Augusta B Lina on Medium]]></description>
        <link>https://medium.com/@augusta.b.lina?source=rss-e16fa9932bcd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*3es4uh7Vaorv_ZkN</url>
            <title>Stories by Augusta B Lina on Medium</title>
            <link>https://medium.com/@augusta.b.lina?source=rss-e16fa9932bcd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:44:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@augusta.b.lina/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Roadmap: AI-Facilitated Collaboration]]></title>
            <link>https://medium.com/@augusta.b.lina/roadmap-ai-facilitated-collaboration-d0e49e8fa0a3?source=rss-e16fa9932bcd------2</link>
            <guid isPermaLink="false">https://medium.com/p/d0e49e8fa0a3</guid>
            <category><![CDATA[ai-democracy]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[ai-safety]]></category>
            <category><![CDATA[aisafetyresearch]]></category>
            <category><![CDATA[cooperative-ai]]></category>
            <dc:creator><![CDATA[Augusta B Lina]]></dc:creator>
            <pubDate>Tue, 30 Jun 2026 07:14:09 GMT</pubDate>
            <atom:updated>2026-06-30T07:14:09.155Z</atom:updated>
            <content:encoded><![CDATA[<p>The following report includes a roadmap into the state of <em>AI-facilitated collaboration</em> across democracy, climate negotiation and frontier model development. We explore the opportunities, challenges, existing work and help define several of the goals and subfields within this area.</p><h3>Background</h3><p>The<strong> Future of Life Foundation</strong> and <strong>Forethought Research</strong> identify collaborative AI as helping diverse groups work together towards shared goals. Automated negotiation is considered a high priority in areas such as diplomatic agreements between nations, democracy or mitigating race dynamics between frontier labs.</p><p><strong>Solutions may include:</strong> deliberation at scale, deal analysts, faithful agents or mediators.</p><p>Existing work tends to use AI as infrastructure for participation. A core objective is to facilitate Pareto-optimal and Nash-bargaining-style fair allocations rather than outcomes driven by leverage and time pressure.</p><h3>Examples</h3><ul><li><strong>Polis </strong>(vTaiwan),<strong> </strong>a digital platform used to collect opinions, facilitate large-scale conversations and consensus building on divisive issues like AI governance.</li><li><strong>ComplexChaos</strong> develops AI tools for stakeholder alignment in complex, high-trust environments like strategic planning and climate negotiations.</li><li><strong>Collective Intelligence Project</strong>, created a roadmap for Democratic AI, runs AI-enabled deliberation and collaborated with Anthropic on democratically trained AI experiments.</li><li><strong>Habermas Machine</strong> (Google Deepmind) designed to help people find common ground on controversial social and political issues. It aims to act as an impartial AI mediator to synthesise diverse opinions into a single, cohesive consensus statement.</li></ul><h3>The Opportunity</h3><h4>Scale with Depth</h4><p>Democratic deliberation (discussion of a subject) and population research is constrained by a scale vs quality trade-off. As the number of participants engaging grows, the depth of exchange declines while smaller groups lack population representativeness.</p><p>Through personalised interfaces and synthesis tools aligned with democratic success metrics, it is possible to increase democratic participation, such as providing public feedback and opinion on policy matters, from dozens to thousands of people.</p><h4>Information Alignment</h4><p>Climate negotiations, for example, can encompass more than 120 agenda items compressed into a few days. Delegates face over 100,000 pages of material, often with limited support and time.</p><p>AI can help delegates become better acquainted with vast corpuses ahead of negotiations and can check ongoing discussions against corpuses in real time, to provide valuable insight and information into feasibility and alignment with original goals.</p><h4>Reducing Information Asymmetries</h4><p>Informational asymmetries often exist between technical experts of various disciplines and/or lay people involved in negotiation. AI has the potential to offer the following gains in real-time:</p><ul><li>Plain-language summaries of technical policy documents</li><li>Fact-checking and source triangulation</li><li>Adaptive explanations</li><li>Translation and accessibility tools</li></ul><h4>Negotiation and Deliberative Feedback Loops</h4><p>Platforms like Thinkscape and Polis already enable discussion participants to see how their positions relate to a broader group in real time, supporting a flow of information that is both intimate and large-scale.</p><p>This can be extended to cross-referencing live discussions against large, diverse corpuses, such as policy databases and economic data, surfacing relevant feedback that informs ongoing negotiations.</p><p>Such tools also make it possible for policymakers to engage in multi-turn exchanges with citizens such as which creates accountability through visible follow-through.</p><h3>Technical Feasibility</h3><p>The technical capabilities to support AI-facilitated collaboration largely exist already. Achieving AI facilitated collaboration from a technical perspective is possible through novel design choices, targeted fine-tuning and other refinement processes.</p><p><strong>Forethought</strong> identifies the following areas developers might focus on:</p><ul><li>Data pipelines</li><li>Developing task-evaluation schemes</li><li>Scaffolding and post-training enhancements</li><li>Compute allocation</li><li>Non-AI barriers like UI, or reputation building</li><li>Complementary technologies</li></ul><p>This may support, for example, deliberation-native AI systems where the training targets are defined by metrics of successful democratic-processes.</p><h3>Deliberation in Democracy</h3><p>AI influences democracy through entry points such as citizen deliberation, a cornerstone of modern democracy.</p><p>It is possible to create complex levers which allow policymakers to engage in multi-turn feedback sessions with citizens and create follow-through accountability.</p><ul><li><strong>Engaged California</strong> statewide deliberative democracy program using AI. Used in the LA fires recovery to process input from thousands of residents.</li><li><strong>UK AI Safety Summit</strong> (2023) An AI system processed input from over 100,000 respondents which was used to inform the Bletchley Declaration, an international agreement on AI safety signed by 28 nations.</li><li><strong>Thinkscape,</strong> an AI-powered collaboration platform which uses swarm technology to support conversations amongst up to 250 people. Smaller groups are continually fed and debate a synthesised version of the larger group’s discussions, allowing for an iteratively updated consensus rather than a single AI synthesis layer.</li></ul><h3>Climate Negotiation</h3><p>In 2025 climate negotiators from nine African countries, representing over 178 million people, used AI to structure priorities and identify common ground. AI was used to generate structured questions, identify overlooked perspectives and synthesise negotiation inputs in real time.</p><p><strong>Outcomes</strong>: 91% of participants reported uncovering insights they would have otherwise missed, coordination time was reduced by 60% and reported empathy rose by 35%.</p><h4>Geneva 10</h4><p>The experiment was a collaboration between the Centre for Multilateral Negotiations and ComplexChaos, through Geneva 10.</p><p>Geneva 10 is a group from ten world leading organisations such as The World Trade Organization, United Nations and UNICEF brought together to prototype AI use-cases in climate negotiations, global trade and cross-border innovation.</p><p>ComplexChaos, an offshoot of the World Economic Forum, aims to scale and build more developments for AI enabled climate deliberation ahead of COP30 Brazil. They are inviting collaboration from governments, NGOs and organisations.</p><h3>Frontier Model Development</h3><p>One objective of AI collaboration is to build the infrastructure such that important AI systems are governed by feedback loops of collective input, known as collective fine-tuning of LLMs.</p><h4>Collective Constitution Project</h4><ul><li>Created by the Collective Intelligence Project (CIP) in collaboration with Anthropic.</li><li>CIP replaced Anthropic’s constitution with one written by over 1,000 Americans. The results suggest that the ‘people-trained’ model was as capable and fairer than those designed solely by researchers.</li></ul><h4>Democratic Inputs to AI</h4><ul><li>OpenAI awarded ten $100,000 grants to fund experiments which set up a democratic process for deciding what rules AI systems should follow. The intention is to encourage the exchange of opinions and facilitate deliberative discussions.</li></ul><h3>Challenges</h3><p>Currently AI safety researchers are largely working in isolation. This has several drawbacks:</p><ul><li><strong>Reputational/Erosion of trust: </strong>Between researchers and negotiation professionals.</li><li><strong>Lack of understanding: </strong>Researchers do not fully understand the nature of negotiation, including its contexts, practical processes and success metrics.</li><li><strong>Mismatched goals: </strong>Many research initiatives focus on automated negotiation, while negotiating bodies tend to focus on AI as participation infrastructure.</li><li><strong>Lack of Technical Expertise in Governance:</strong> Currently governance bodies do not have the sufficient in-house expertise to properly procure, monitor or implement deliberation tools.</li></ul><p>These gaps lead to mistrust, delayed adoption, and wasted resources. They also leave deliberators dependent on private vendors whose incentives may not align with democratic objectives which makes it easy for poorly designed tools to undermine the processes they’re meant to support. For example, a key technical risk is algorithmic bias in information synthesis or tools that fail to generate fair allocations while masking this through opaque processes.</p><p>This points to the need for unified teams of researchers and deliberators, a standard which is being set by ComplexChaos, which aims to fully understand the context and success metrics of negotiation and deliberation professions.</p><h3>Funding</h3><h3>Advice for Funders</h3><p>CEIP recommends that funders prioritise multi-year projects and require grantees to adopt negotiation-informed quality benchmarks as a condition of funding.</p><h3>Philanthropic &amp; Venture Bodies</h3><p><strong>Macroscopic Ventures </strong>A Swiss non-profit which aims to build a future guided by reason and compassion. Backed the Cooperative AI Foundation with a $15 million investment.</p><p><strong>Village Global </strong>ComplexChaos is backed by Village Global, a VC chaired by Reid Hoffman, and funded by Bill Gates, Mark Zuckerberg, and Jeff Bezos, indicating that major players in tech are willing to invest.</p><p><strong>Humanity AI </strong>A $500 million collaborative grantmaking initiative with a research priority on AI and democracy.</p><p><strong>Ford Foundation, Omidyar Network, OneProject, Survival and Flourishing Fund, Amaranth Foundation </strong>Funders of the Democratic AI Roadmap from the Collective Intelligence Project.</p><h3>Fellowships</h3><ul><li><strong>OpenAI, Democratic Inputs to AI</strong>: $100,000 (12–16 weeks)</li><li><strong>Cooperative AI Research Fellowship</strong>: $12,000 (12 weeks)</li><li><strong>Future of Life Foundation, Incubator Fellowship on AI for Human Reasoning</strong>: $25,000–$50,000 (12 weeks)</li><li><strong>Mozilla Foundation, Democracy x AI Fellowship</strong>: $50,000 (12 months), $250,000 follow-on</li><li><strong>University of Virginia Digital Technology for Democracy, AI &amp; Democracy Grants</strong>: $50,000 (12 months)</li></ul><p>Fellowship values, typically $12,000–$100,000, indicate the precedent of seed investment per project. Though full project costs are higher once operational overheads are included.</p><h3>Goals for the Next 12 Months</h3><p>Below are recommendations for the next 12 months and an example of what a deliverable might look like.</p><p><strong>Bridge the researcher-practitioner gap. </strong>For example, embedded fellowships where AI researchers sit inside negotiation bodies.</p><ul><li>Establish 2 + fellowships to start in the next 6–18 months.</li><li>Encourage existing fellowships to update their selection strategy.</li></ul><p><strong>Increase the real-world prototypes</strong>. Support the development of prototypes within real-world contexts such as conferences.</p><ul><li>Establish 3+ prototypes for implementation in the next 12–24 months.</li></ul><p><strong>Create a centralised information source</strong>. Much information on AI facilitated collaboration is disparate and difficult to locate. Help establish a single organisation as a central source of information which other bodies may refer to when building out their own negotiation systems.</p><ul><li>Reach an agreement with a body on this e.g. Collective Intelligence Project, and facilitate funding for this agenda.</li></ul><p><strong>Build technical capacity within governance bodies. </strong>Facilitate the hiring of full-time technical staff within deliberation organisations and provide support for agenda setting.</p><ul><li>Support the development of in-house AI departments within government bodies, including hiring, integration and agenda setting</li></ul><p><strong>Create investment standards.</strong> Work with funders and major diplomatic/democratic bodies to create standards for investment in AI facilitated democratic and collaboration processes.</p><ul><li>Work with various deliberation bodies to build these standards.</li><li>Gain buy-in from 2+ funders</li></ul><h3>Empirical Research &amp; Planning</h3><p>The above research was conducted through online sources. The next step would be to engage with professionals working in the space to build a living strategy document and agenda. The below section lays out questions one might ask when planning specific initiatives based on this research.</p><h3>Part A: Contact Experts</h3><ul><li>People <em>encountering</em> the issue — hear their problems (needs)</li><li>People <em>addressing</em> the issue — their solutions (opportunities)</li><li>Trial ideas to experts — collect feedback</li></ul><h3>Part B: Scope</h3><p>Scope the project’s position and possible implementation.</p><h4>Project Position</h4><ul><li>What are the initiative’s research focus points?</li><li>What is the initiative’s position within the community?</li><li>What is the impact — how will it help?</li><li>What are key deliverables?</li></ul><h4>Implementation</h4><p><strong>Market</strong></p><ul><li>Who is the market? E.g. specific actors within government bodies.</li><li>What are the barriers to market?</li></ul><p><strong>Pathways</strong></p><ul><li>What is a small, medium, and complex version of this solution?</li><li>What are each of the below group’s opinions on each version?</li><li>Users: People working on the issue, people researching the issue</li><li>What would each version take to deliver in terms of cost, time, complexity? Are there unique challenges (e.g. requires access to high-profile individuals)?</li></ul><h4>Small Test</h4><ul><li>What is the smallest possible test we can run to check our assumptions and indicate whether the idea might fail?</li><li>Does it check technical assumptions?</li><li>Does it check qualitative assumptions?</li><li>What is our turnaround time and budget to implement the test?</li><li>Who are the best people to implement it? (in-house, recruit, fellowship, etc.)</li><li>What are our success metrics?</li><li>What is the most likely reason the test won’t work e.g. give a misleading result, or fail to materialise? What can we do to prevent that?</li></ul><p><strong>Execute the small test.</strong></p><p><strong>Write the conclusion:</strong></p><ul><li>Assumptions — which held?</li><li>Is the initiative feasible?</li><li>Product spec</li><li>Limitations of review</li><li>Possible next steps</li></ul><p><strong>Next steps:</strong> e.g. scaling up impact, complexity, and reach.</p><p><em>The above report is a live document. If you feel facts were mistakenly represented, please reach out at augusta.b.lina[at]gmail[dot]com.</em></p><p><strong><em>AI Declaration:</em></strong><em> Some background research was conducted with the help of Claude. However, all writing and ideas are original.</em></p><p><strong>Sources</strong></p><p>Vaintrob, L., Cotton-Barratt, O. (2025).<a href="https://www.forethought.org/research/ai-tools-for-existential-security"> AI Tools for Existential Security</a></p><p>Future of Life Foundation. (2025).<a href="https://flf.org/projects/ai-facilitated-collaboration/"> AI-Facilitated Collaboration</a></p><p>Noveck, B.S., Martí, J.L. (2026).<a href="https://rebootdemocracy.ai/blog/habermolt"> In Lobsters We Do Not Trust: The Wrong Path for AI and Democracy</a></p><p>Sourbut, O. (2025).<a href="https://forum.effectivealtruism.org/posts/dQWHB2s3frDjXwGFe/funded-fellowship-ai-for-human-reasoning-fellowship-with-the"> FLF Fellowship on AI for Human Reasoning: $25–50k, 12 weeks</a></p><p>The FLF Team. (2026).<a href="https://flf.org/coordination/"> AI-Facilitated Coordination</a></p><p>RebootDemocracy.AI. (2025).<a href="https://rebootdemocracy.ai/blog/habermas-machine-%20workshop"> The Peacemaking Machine: Can AI Improve Democratic Deliberation?</a></p><p>Lorsch, T., Ben Dror, M. (2025).<a href="https://www.weforum.org/stories/2025/09/ai-diplomacy-scale-inclusion-global-climate-negotiations/"> How AI Helps Diplomats Scale Inclusion in Global Climate Negotiations</a></p><p>Weinberg, M. (2026).<a href="https://carnegieendowment.org/research/2026/05/realizing-the-potential-gains-of-ai-enabled-deliberative-democracy"> Realizing the Potential Gains of AI-Enabled Deliberative Democracy</a></p><p>George, R., Klaus, I. (2026).<a href="https://carnegieendowment.org/research/2026/01/ai-and-democracy-mapping-the-intersections"> AI and Democracy: Mapping the Intersections</a></p><p>UK Department for Science, Innovation and Technology. (2023).<a href="https://www.gov.uk/government/publications/international-survey-of-public-opinion-on-ai-safety"> International Survey of Public Opinion on AI Safety</a></p><p>OpenAI. (2023).<a href="https://openai.com/index/democratic-inputs-to-ai/"> Democratic Inputs to AI</a></p><p>OpenAI. (2023).<a href="https://vimeo.com/875039398"> Democratic Inputs to AI Demo Day</a> [Video]</p><p>Digital Technology for Democracy Lab, University of Virginia. (2026).<a href="https://e2uaspmb7dd.exactdn.com/wp-content/uploads/2026/02/2026-DTD-AI-Democracy-Grant-program.pdf"> AI &amp; Democracy Grant Program</a></p><p>Humanity AI. (2025).<a href="https://humanityai.ai/what-we-do/"> What We Do</a></p><p>Collective Intelligence Project. (2024).<a href="https://static1.squarespace.com/static/631d02b2dfa9482a32db47ec/t/65f9a1296f1a357e918f7a58/1722968408230/CIP_+A+Roadmap+to+Democratic+AI.pdf"> A Roadmap to Democratic AI</a></p><p>Thinkscape.<a href="https://www.thinkscape.ai/"> thinkscape.ai</a></p><p>Village Global.<a href="https://www.villageglobal.com/"> villageglobal.com</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d0e49e8fa0a3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Custom Gym Environment for a Quantum Error Correction Decoder (Logical Memory)]]></title>
            <link>https://medium.com/@augusta.b.lina/a-custom-gym-environment-for-a-quantum-error-correction-decoder-logical-memory-750ae84217d3?source=rss-e16fa9932bcd------2</link>
            <guid isPermaLink="false">https://medium.com/p/750ae84217d3</guid>
            <category><![CDATA[quantum-computing]]></category>
            <category><![CDATA[quantum-error-correction]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[actor-critic]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Augusta B Lina]]></dc:creator>
            <pubDate>Wed, 06 Aug 2025 13:09:43 GMT</pubDate>
            <atom:updated>2025-08-10T16:12:38.174Z</atom:updated>
            <content:encoded><![CDATA[<p>This is the second article in a series on research I’m conducting into building an actor-critic reinforcement learning (RL) decoder for the surface code of a logical qubit. In this article we create a custom RL environment using <a href="https://github.com/openai/gym/tree/v0.21.0">OpenAI’s Gym</a> library to train a decoding algorithm for a quantum error correcting topological code.</p><p>A foundation of great research is a good ecosystem, both in terms of theory and resouces for implementation. Quantum Error Correction (QEC) has an incredible amount of theory available but only an emergent ecosystem of implementation resources. This is particularly true of deep learning QEC. RL is a flexible, non-analytical method which makes it potentially well suited to the dynamic nature of surface code topology in computational environments.</p><p>Many large-scale, real-world implementations of deep learning systems such as Google’s <a href="https://www.nature.com/articles/s41586-024-08148-8">AlphaQubit</a> or <a href="https://arxiv.org/abs/2412.19437">LLMs</a> tend to use multi-stage learning pipelines with several different statistical and neural network-based phases. One approach tends to not be sufficient and data may be processed through a variety of statistical, supervised, and RL methods in order to achieve a system capable of handling complex inputs and outputs. For this reason, it’s important for researchers to be able to experiment in a way that allows them to build complex systems capable of multiple processing stages.</p><p>My research has focused on the RL decoding problem, an area of research which is interesting enough to fill a PhD. There is a lot of available theory in both QEC and RL which could be brought together for very interesting results. But this is challenging to do if the foundational implementation resources are not available for researchers to use and it is wasted energy for researchers to reinvent the wheel. While technical problems can often be straightforward once broken down into smaller components, identifying how to decompose a problem can be challenging.</p><p>For this reason, I am sharing my research to provide a foundation for future researchers to build upon to generate more complex RL environments for the decoding problem.</p><p>The solution proposed here builds upon the foundations laid out in the first article, which breaks the quantum system logic into smaller pieces. We then implement a straightforward Markov Decision Process (MDP) and RL logic to bring together a custom RL environment which can be used by other researchers to train a wide range of RL algorithms.</p><p><strong>The mapping and training process can be thought of in four stages of which this article covers the second:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2uur4Six3r4hor3s3PkHVg.png" /></figure><p><strong>Gym is an </strong><a href="https://github.com/openai/gym/tree/v0.21.0"><strong>open source</strong></a><strong> Python library</strong> which provides an API for communicating between RL algorithms and environments. For our purposes, it provides the functionality for us to construct a custom RL environment. In the next article, I will discuss building an actor-critic RL algorithm with supervised pretraining using data generated using Minimum Weight Perfect Matching (MWPM).</p><p><strong>Background</strong></p><p>This series focuses on coding implementation details. However, for more information on quantum error correction (QEC) theory, this free <a href="https://www.coursera.org/learn/quantum-error-correction">course</a> published by Google is well regarded. Additionally, if you have not read the first article in this series, <a href="https://medium.com/@augusta.b.lina/building-a-quantum-simulation-system-for-a-reinforcement-learning-environment-3b1046f2af4d"><em>Building a Quantum Simulation System for a Reinforcement Learning Environment</em></a>, I recommend you start there. The discussion in this article presupposes an understanding of the code logic already established. As before, the objective of this article is to be succinct. I will release more detailed information in the future for those interested.</p><h4><strong>Overview</strong></h4><p>Below is the full code for the environment. We will discuss this in smaller code blocks.</p><pre>class LQ_DecoderEnv(gym.Env):<br>    def __init__(self, dist=3, p=0.01, rf=0.1):<br>        super().__init__()<br>        # Initialise the key objects from the qecsim library &amp; additional objects<br>        self.code = PlanarCode(dist, dist) # quantum circuit<br>        self.error_model = DepolarizingErrorModel() # error model<br>        self.p = p # probability of an error<br>        self.stabilizers = self.code.stabilizers # extract stabiliser attribute<br>        self.reward_factor = rf # set reward factor - used to in reward weighting<br>        self.rng = np.random.default_rng(10) # generate new random number - for error generation<br>        self.logical_operators = self.code.logicals # extract logical operators using qecsim logicals attribute<br><br>        # Observation = syndrome (length = len(stabilizers))<br>        self.observation_space = spaces.MultiBinary(len(self.stabilizers))<br><br>        # Action = recovery operator (full BSF, length = 2 * n_data)<br>        # Generate an error to determine the length of the action vector<br>        self.error_dummy = self.error_model.generate(self.code, self.p, self.rng)  # this error is not used to compute a syndrome but to initialise the error vector<br>        self.vec_length = len(self.error_dummy) # Extract the length of the generated error<br>        self.action_space = spaces.MultiBinary(self.vec_length) # Use the length of the error vector to determine the length of the action space vector<br><br>        self.error = None<br>        self.syndrome = None<br>        self.transition_history = None<br>        self.t = None<br><br>    def reset(self):<br>        # calculate the initial state the agent starts from.<br>        # In this instance, this is an empty syndrome vector generated from the unchanged original circuit.<br><br>        # Initialise transition history<br>        self.t = 1 # time steps (we index from 1)<br>        self.transition_history = np.zeros(self.vec_length, dtype=np.int64) # initialise transition history to zeros # length: transition _history = recovery = error<br>        # compute syndrome<br>        self.syndrome = pt.bsp(self.transition_history, self.stabilizers.T)<br>        # Reset RL parameters (done = False, time steps = 1)<br>        return self.syndrome<br><br>    def step(self, recovery):<br>        # Apply State Transition Algorithm (error -&gt; SH update -&gt; syndrome -&gt; recovery -&gt; SH update -&gt; logical error test)<br><br>        # Update transition history with the action (recovery)<br>        self.transition_history ^= recovery # update transition history = recovery ⊕ error<br>        <br>        # Apply Reward<br>        self.syndrome = pt.bsp(self.transition_history, self.stabilizers.T) # generate new syndrome vector<br>        # if a latent underlying error exists, we penalise the agent<br>        latent = 1 if np.any(self.syndrome) else 0<br>        reward = (self.reward_factor*self.t)**2 - latent # the reward is based on the number of time steps and the whether a latent error exists<br><br>        info = {&quot;timestep&quot;: self.t}<br><br>        # Generate New State Transition &amp; Observation<br>        self.t += 1 # increment time step<br><br>        # Generate new error<br>        self.error = self.error_model.generate(self.code, self.p, self.rng)  # generate new error vector<br>        # Update transition history<br>        self.transition_history ^= self.error<br>        # Compute new syndrome<br>        self.syndrome = pt.bsp(self.transition_history, self.stabilizers.T) # generate syndrome vector<br>        # test for logical error<br>        # returns an array [a,b] corresponding to an X and Z logical<br>        # If either index the 1, this line returns TRUE and stores it in &#39;done&#39;<br>        done = bool(np.any(pt.bsp(self.transition_history, self.logical_operators.T)))<br><br>        return self.syndrome, reward, done, info</pre><h4><strong>Code Structure</strong></h4><p>Let’s first review the structure. The custom environment inherits from Gym’s Env class, which requires the <em>__init__</em>, <em>reset</em>, and <em>step</em> methods. These define the environment dynamics and return an experience tuple in the form (<em>state, reward, done, info</em>).</p><pre>class LQ_DecoderEnv(gym.Env):<br>    def __init__(self, dist=3, p=0.01, rf=0.1)<br>    # [CODE]<br><br>    def reset(self):<br>    # [CODE] <br><br>    def step(self, recovery):<br>    # [CODE]<br><br>        return self.syndrome, reward, done, info</pre><p><strong>This environment accepts three arguments: </strong>the code distance (<em>dist</em>), the error probability (<em>p</em>), and a reward scaling factor (<em>rf</em>). These are specified when the environment is instantiated as part of the RL training loop.</p><p><strong>The __init__ method </strong>defines the Python objects that underpin the MDP logic of the environment. These include tunable parameters and objects from the <em>qecsim</em> library, such as the surface code structure.</p><p><strong>The <em>reset</em> method</strong> is called at the beginning of each episode and initialises the environment state.</p><p><strong>The <em>step</em> method </strong>defines the MDP transition dynamics. It applies a new error, generates a syndrome, updates the environment state, calculates the reward and checks for episode termination. This produces the experience tuple (<em>state, reward, done, info</em>) which is used for training. When we call the <em>step</em> method during training, we pass the recovery action which is generated by the actor-critic algorithm.</p><h4><strong>__init__ Method</strong></h4><p>This method initialises the key components of our environment. We begin by creating a planar surface code using <em>qecsim</em>’s <em>PlanarCode</em> object. This defines the layout of data and ancilla qubits, as well as the stabiliser generators and logical operators of the code. We also instantiate the error model, <em>DepolarizingErrorModel</em>, which simulates random Pauli 𝑋, Y, and 𝑍 errors applied to the data qubits with probability p/3.</p><p>The stabilisers are extracted from the code object using <em>self.code.stabilizers</em>, and represent the check operators used for syndrome extraction. The logical operators are extracted using<em> self.code.logicals </em>these<em> </em>will be used to check whether a logical failure has occurred. The <a href="https://medium.com/@augusta.b.lina/building-a-quantum-simulation-system-for-a-reinforcement-learning-environment-3b1046f2af4d">first article</a> in this series covers the information you need to understand the first code block within the <em>__init__</em> method so we will not go into further detail here.</p><p><strong>State Space</strong></p><p>In Gym, we must define the structure of the observation space and action space. Both are represented in this environment as binary vectors using <em>spaces.MultiBinary()</em>. The observation, the state, is the syndrome, which is a binary vector of length equal to the number of stabilisers. Each element of the syndrome corresponds to a different stabiliser and indicates whether that stabiliser commutes or anticommutes with the circuit in its current state. As a refresher, the current state is represented as the initialised circuit with the transition history operator applied to it. A value of 0 means the stabiliser commutes, while a 1 indicates that it anticommutes, meaning that a 1 indicates a parity violation, or detection event. By examining the pattern of 0s and 1s across the syndrome, we can infer information about the underlying error. Specifically, what type of Pauli error may have occurred and on which qubits.</p><p><strong>Action Space</strong></p><p>The action space is the recovery operation generated by the agent. It is encoded in binary symplectic form which means that it is twice as long as the number of qubits we are applying recovery operations to. The first half of the vector indicates whether we apply an X correction to a particular qubit or not. The second half indicates whether we apply a Z correction. A discussion on how the binary symplectic form is represented in code can be found <a href="https://qecsim.github.io/api/paulitools.html?highlight=binary%20symplectic%20vector">here</a> in the <em>qecsim</em> documentation. To determine this length, we generate a dummy error vector from the error model, then extract its length to define the action space.</p><pre>    def __init__(self, dist=3, p=0.01, rf=0.1):<br>        super().__init__()<br>        # Initialise the key objects from the qecsim library &amp; additional objects<br>        self.code = PlanarCode(dist, dist) # quantum circuit <br>        self.error_model = DepolarizingErrorModel() # error model<br>        self.p = p # probability of an error <br>        self.stabilizers = self.code.stabilizers # extract stabiliser attribute<br>        self.reward_factor = rf # set reward factor - used to in reward weighting<br>        self.rng = np.random.default_rng(10) # generate new random number - for error generation<br>        self.logical_operators = self.code.logicals # extract logical operators using qecsim logicals attribute <br><br>        # Observation = syndrome (length = len(stabilizers))<br>        self.observation_space = spaces.MultiBinary(len(self.stabilizers))<br><br>        # Action <br>        # Generate an error to determine the length of the action vector<br>        self.error_dummy = self.error_model.generate(self.code, self.p, self.rng)  # this error is not used to compute a syndrome but to initialise the error vector<br>        self.vec_length = len(self.error_dummy) # Extract the length of the generated error<br>        self.action_space = spaces.MultiBinary(self.vec_length) # Use the length of the error vector to determine the length of the action space vector<br><br>        self.error = None  <br>        self.syndrome = None<br>        self.transition_history = None<br>        self.t = None</pre><p><strong>The <em>reset</em> Method</strong></p><p>The <em>reset</em> method creates a clean quantum memory with no errors. At the start of each episode we set the number of time steps to 1 and we initialise a transition history vector of all zeros. This is, in effect, applying the identity operator to the initialised code.</p><p>We generate a syndrome using the transition history, which is also a vector of all zeros because the transition history is an identity operation and therefore we have no detection events. The <em>reset</em> method then returns the syndrome which becomes the first state received by the agent.</p><pre>    def reset(self):<br>        # calculate the initial state the agent starts from. <br>        # In this instance, this is an empty syndrome vector generated from the unchanged original circuit.<br><br>        # Initialise transition history<br>        self.t = 1 # time steps (we index from 1)<br>        self.transition_history = np.zeros(self.vec_length, dtype=np.int64) # initialise transition history to zeros # length: transition _history = recovery = error<br>        # compute syndrome<br>        self.syndrome = pt.bsp(self.transition_history, self.stabilizers.T)<br>        # Reset RL parameters (done = False, time steps = 1)<br>        return self.syndrome</pre><p><strong>The <em>step</em> Method</strong></p><p>The <em>step()</em> method defines the transition dynamics of the environment. It implements the logic for one time step in the MDP. Each step runs a single run of the quantum circuit which represents the topological code. This includes a single round of error injection, generating a syndrome, applying a recovery operation by the agent and evaluating whether a logical error has occurred. This is shown in the diagram below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cfOyOhefJjOVGUiEfSHRUA.png" /></figure><p>Within the code for the environment this process can be broken down into nine steps:</p><ol><li>The agent applies the recovery to the transition history</li><li>Generate a new syndrome</li><li>Compute the reward</li><li>Increment the time step <em>t</em></li><li>Sample a new error from the noise model</li><li>Update the transition history with the new error</li><li>Compute new syndrome (Observation)</li><li>Check for logical error (episode termination)</li><li><em>Return the experience tuple</em></li></ol><p>These steps have been numbered in the in-line commentary of the below code block to help readers identify where they are applied.</p><pre>def step(self, recovery):<br>        # Apply State Transition Algorithm <br><br>        # ACTION<br>        # 1. Update transition history with the action (recovery)<br>        self.transition_history ^= recovery # update transition history = recovery ⊕ error<br>        <br>        # REWARD<br>        # 2. Generate syndrome<br>        self.syndrome = pt.bsp(self.transition_history, self.stabilizers.T) # generate new syndrome vector<br>        # 3. Calculate reward<br>        # if a latent underlying error exists, we penalise the agent<br>        latent = 1 if np.any(self.syndrome) else 0<br>        reward = (self.reward_factor*self.t)**2 - latent # the reward is based on the number of time steps and the whether a latent error exists<br><br>        info = {&quot;timestep&quot;: self.t}<br><br>        # STATE TRANSITION <br>        # 4. Increment the time step<br>        self.t += 1 # increment time step<br>        <br>        # 5. Generate new error<br>        self.error = self.error_model.generate(self.code, self.p, self.rng)  # generate new error vector<br>        # 6. Update transition history<br>        self.transition_history ^= self.error<br><br>        # NEW OBSERVATION<br>        # 7. Compute new syndrome<br>        self.syndrome = pt.bsp(self.transition_history, self.stabilizers.T) # generate syndrome vector<br>        <br>        # TERMINAL STATE EVALUATION<br>        # 8. test for logical error<br>        # returns an array [a,b] corresponding to an X and Z logical<br>        # If either index the 1, this line returns TRUE and stores it in &#39;done&#39;<br>        done = bool(np.any(pt.bsp(self.transition_history, self.logical_operators.T)))<br><br>        return self.syndrome, reward, done, info</pre><p><strong>Reward</strong></p><p>We define success as the lifetime of the logical qubit. That is the number of time steps before a logical error occurs. A reward is given based on both the number of time steps survived and whether any residual detection events arise in the syndrome after recovery. The timestep term of the reward increases quadratically with time, i.e. <em>t²</em>, in order to increase the value of time as it progresses. This adds an additional incentive compared to a linearly increasing reward.</p><p>A penalty of -1 is applied if a detection event remains in the syndrome after recovery. The latent error penalty is an intermediary reward to gently guide the agent to realise that correcting latent errors, i.e. errors that occur on underlying physical qubits but do not corrupt the logical qubit, can help prevent a logical error from occurring.</p><p>The <em>rf</em> factor is used to scale up and scale down the weighting of the time step reward compared to the latent error penalty. This is a useful parameter to fine-tune.</p><p><strong>The experience tuple returned includes:</strong></p><ul><li><strong>State:</strong> In a binary array representing a single syndrome measurment</li><li><strong>Reward:</strong> 0.1<em>t² — latent</em></li><li><strong>Done: </strong>Boolean signal for logical error.</li><li><strong>Info: </strong>timestep index <em>t</em></li></ul><p><strong>Conclusion</strong></p><p>And that concludes our discussion on a custom Gym environment for training a decoding algorithm. There is an incredible amount of good-quality theoretical research available which can be used to extend this environment to either create a more complex quantum system that better reflects experimental settings of topological quantum codes, or which might have additional fault tolerant properties such as multiple samples per round, or feeding the agent syndrome histories in addition to the present slice. I will release more information on the algorithm used to train an agent in this environment soon, and I hope that other researchers find this series helpful.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=750ae84217d3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Quantum Simulation System for a Reinforcement Learning Environment]]></title>
            <link>https://medium.com/@augusta.b.lina/building-a-quantum-simulation-system-for-a-reinforcement-learning-environment-3b1046f2af4d?source=rss-e16fa9932bcd------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b1046f2af4d</guid>
            <category><![CDATA[decoder]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <category><![CDATA[quantum-error-correction]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Augusta B Lina]]></dc:creator>
            <pubDate>Mon, 04 Aug 2025 08:51:01 GMT</pubDate>
            <atom:updated>2025-08-09T10:21:28.170Z</atom:updated>
            <content:encoded><![CDATA[<p>This is the first article in a series on research I’m conducting into building an actor-critic reinforcement learning (RL) decoder for the surface code of a logical qubit. This article covers how to construct a simulation system that functions as the environment an agent, the decoding algorithm, can learn from. In later articles in this series we will discuss how to wrap this environment in <a href="https://medium.com/r?url=https%3A%2F%2Fgithub.com%2Fopenai%2Fgym%2Fblob%2Fv0.21.0%2Fgym%2Fcore.py">OpenAI’s Gym wrapper</a>, construct a deep learning architecture for this problem, and build a custom decoder class — tying the moving parts of this research together.</p><p>Many of the quantum simulation packages available like <a href="https://github.com/quantumlib/Stim">Stim</a> do not have the functionalities which allow us to construct a decoder base class which can be integrated into the existing simulation functionalities needed to train an agent. Therefore, the first step in my research was to build a simulation system that was modular, allowing me to extract the reward, state and episode termination signal at each time step, and for which I could build a custom decoder. To do this, I have used a Python package called <em>qecsim</em> by David K. Tuckett (<a href="https://github.com/qecsim/qecsim/tree/master">GitHub</a> &amp; <a href="https://qecsim.github.io/usage.html">Documentation</a>).</p><p>My objective in this tutorial is to keep the implementation discussion as clear and succinct as possible. I will release more detailed information in the future for those interested. Some underlying information is indicated using italics though it is possible to work through this article without understanding these details.</p><p>As with much research, the answer is quite simple but takes a long time to derive for the first time. Therefore I am sharing my work to make this easier for other researchers. To start, we define what a single round of the simulation system might look like.</p><p>Below is a the flow of a single round of a quantum circuit. In this system we focus on a circuit which encodes logical qubit information for quantum memory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jfii6QfGkcMioqQNmV3h_g.png" /></figure><p>In the next diagram we have highlighted which parts of this circuit system map to a RL framework.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cfOyOhefJjOVGUiEfSHRUA.png" /></figure><blockquote><strong>Aside:</strong> qecsim has a run() command which runs a circuit many times but it generates an aggregate result. This makes it useful for generating expected metrics but not for training a deep learning system which requires the state, action and reward for each time step.</blockquote><h4>1. Initialise Components</h4><p>In the first round, we initialise the key components used in the quantum system. First we generate the quantum circuit which will encode the logical qubit and for which we wish to build the decoding algorithm. In this example we use a distance 3 planar code and depolarising noise. We set the error probability to 0.01 meaning:</p><ul><li>There is a 1% chance of an error occurring</li><li>A 1/3% chance of an X, Z and Y error respectively i.e. approximately 0.0033%</li></ul><pre># Packages and Dependencies <br>pip install -q qecsim<br>import numpy as np<br>from qecsim import paulitools as pt<br>from qecsim.models.generic import DepolarizingErrorModel<br>from qecsim.models.planar import PlanarCode, PlanarMWPMDecoder<br><br># Initialise System<br>circuit = PlanarCode(3, 3)<br>error_model = DepolarizingErrorModel()<br>error_probability = 0.01</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q0O0d9T-hBxSR0GTcs6GWg.png" /></figure><p>In <em>qecsim</em> the error applied is defined by a random number so we also initialise a random number generator.</p><pre>rng = np.random.default_rng(seed=10)</pre><p>Before building a reinforcement learning decoder, we need to define a quantum circuit which satisfies our requirements for training. For development purposes, we use a built-in decoder from <em>qecsim</em> which uses minimum weight perfect matching (MWPM). This will be switched out for a trained decoder in future phases of the research. We also extract the stabilisers from the circuit which will be used for syndrome generation. These are generated automatically by <em>qecsim</em> from the built-in planar code.</p><pre>decoder = PlanarMWPMDecoder()<br>stabilizers = circuit.stabilizers</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PMCZhREdhS9k5SVNbcHCvg.png" /></figure><h4>2. Generate Error</h4><p>We can generate an error on the circuit using a single line of code.</p><pre>error = error_model.generate(circuit, error_probability, rng)</pre><blockquote>For those interested, the error is represented as a numpy array representing a binary symplectic vector of X and Z operations.</blockquote><p>We then store the error vector in a new variable called <em>transition_history</em>. This will be used for us to track and apply the cumulative state transitions to the circuit. The transitions history records all changes to the underlying environment (the circuit) since it was initialised.</p><p>In the second diagram we can see that state transitions occur when we generate an error on the circuit or when we apply a recovery operation. Both error and recovery operations are represented in the same way, using a numpy array to reflect the <a href="https://qecsim.github.io/api/paulitools.html?highlight=binary%20symplectic%20form">binary symplectic form (BSF)</a>.</p><pre>transition_history = error</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R7BhKpClEctlPILPiSUlog.png" /></figure><h4>3. Extract Syndrome (State)</h4><p>In qecsim the syndrome is calculated using a single line of code with the generated error and the transpose of the circuit stabilisers being passed to the function<em> </em><a href="https://qecsim.github.io/api/paulitools.html?highlight=binary%20symplectic%20product"><em>pt.bsp()</em></a> which performs the <a href="https://qecsim.github.io/api/paulitools.html?highlight=binary%20symplectic%20product">binary symplectic product (BSP)</a>. To implement this effectively you do not need to understand the BSP but more information can be found <a href="https://qecsim.github.io/overview.html">here</a>. This syndrome constitutes the state of the RL algorithm. It will be used at each time step by the agent to identify a recovery operation.</p><pre>syndrome = pt.bsp(error, stabilizers.T)</pre><blockquote>From a theoretical perspective, the syndrome is generated by checking whether the transition history commutes with the stabiliser operators which is calculated by the BSP between the transition history and the transpose of the stabilisers.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SIr912KgNQqbWAAiMQM9Cg.png" /></figure><h4>4. Generate Recovery Action</h4><p>For now we are able to generate a recovery operation using a single line of code which receives the circuit and the syndrome information as arguments. This is because we are using an existing decoder function to generate the recovery operations based on the syndrome information.</p><p>We will later need to define a custom decoder based on the <em>qecsim</em> decoder base class. This will allow us to integrate a deep learning decoder into the quantum simulation system we are currently defining. But for now, it is important to have a functional environment which we can build a RL system upon and this line of code will serve as a placeholder while we structure the flow of information in our simulation system.</p><pre>recovery = decoder.decode(circuit, syndrome)</pre><p>As previously mentioned, the output of the recovery and error operations are numpy arrays representing the binary symplectic form. The implications of this on the development of the neural network will be explored in another article.</p><p>By using the Python operator ^= we are applying the XOR operation between the recovery operation and the transition history element-wise which is represented as:</p><blockquote>recovery ⊕ transition history</blockquote><p>This is equivalent to the below for the first round of the recovery operation.</p><blockquote>recovery ⊕ error</blockquote><p>By iteratively applying recovery operations to the transition history, we obtain a binary symplectic vector that compactly represents the cumulative effect of all operations applied to the quantum circuit since it was initialised. In stabiliser codes and quantum error correction (QEC), this technique allows us to track how Pauli errors and corrections compose over time. For example, applying a Pauli operator such as Z, X, or Y twice on the same qubit results in the identity, so an error like ZXI followed by itself reduces to III (i.e. no operation). In simple language, this can be thought of similarly to cancelling out during division e.g. 2/2 = 1. This compactness of notation is the basis of much stabiliser and QEC theory and is a helpful tool here for tracking the cumulative state of the underlying quantum circuit. Thanks to <em>qecsim</em> we can represent this cumulative activity quite easily by applying one vector to another as an XOR operation.</p><p>Updating the compact notation of the cumulative operations performed on the original quantum circuit is our general approach for updating the transition history. We apply either the recovery operation or error using ^= to the transition history.</p><p>At any point, the transition history helps us store the underlying state of the circuit and this forms the heart of our simulation system. We can confirm at any point whether the transition history commutes with the stabilisers as we did in <em>Section 3 </em>to generate the syndrome for the underlying state which will be the result of a mixture of errors and imperfect recovery operations. For now, we will just update the transition history as described:</p><pre>transition_history ^= recovery</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oH3fUf_rdJR9wQmKsgm7YQ.png" /></figure><h4>5. Inspect for Logical Errors</h4><p>The final part of a single round within the quantum system is to check for logical errors after applying the recovery action. If a logical error has occurred the system has failed and the logical information has been corrupted. In RL we define a logical error as causing the end of an episode. This can be calculated by using<em> pt.bsp()</em> to calculate the BSP between the transition history and the transpose of the logical operators which we can extract as an attribute from <em>qecsim’s</em> planar code using the code <em>circuit.logicals</em>.</p><pre>pt.bsp(transition_history, circuit.logicals.T)</pre><p>The output of this code is an array of length 2 in binary format.e.g. [0 0]. The first entry tells us whether a logical X failure has occurred and the second whether a logical Z failure has occurred. It is worth noting that the rewards will be based on the number of time steps the logical qubit remains alive. i.e. the lifetime of the logical qubit.</p><p>This core algorithm gives us the building blocks upon which to define a neural network architecture and the implementation details of a RL algorithm. Thanks to <em>qecsim</em> and some Python we are now able to run a cumulative system which allows us to extract our state, reward and terminal state signal at each time step. We will start to bring everything together in later code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b1046f2af4d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>