<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Engin Yapici on Medium]]></title>
        <description><![CDATA[Stories by Engin Yapici on Medium]]></description>
        <link>https://medium.com/@enginyapici?source=rss-a009c1fc1b90------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*wFp9jC4BalwahJ-jwYg7kQ.jpeg</url>
            <title>Stories by Engin Yapici on Medium</title>
            <link>https://medium.com/@enginyapici?source=rss-a009c1fc1b90------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 23:06:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@enginyapici/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[PPIFlow: Picomolar Binder Design and the Limits of the BLI Data]]></title>
            <link>https://medium.com/@enginyapici/ppiflow-picomolar-binder-design-and-the-limits-of-the-bli-data-8a0336bf526a?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a0336bf526a</guid>
            <category><![CDATA[antibody-design]]></category>
            <category><![CDATA[protein]]></category>
            <category><![CDATA[ai-in-biology]]></category>
            <category><![CDATA[protein-engineering]]></category>
            <category><![CDATA[protein-design]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 16 Jun 2026 14:51:14 GMT</pubDate>
            <atom:updated>2026-06-16T14:51:14.833Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://www.cpl.ac.cn/en/"><em>Changping Laboratory</em></a><em>’s flow-matching pipeline reports picomolar affinities for six of seven mini-binder targets, all measured by BLI.</em></p><p>Changping Laboratory released <a href="https://www.biorxiv.org/content/10.64898/2026.01.19.700484v2">PPIFlow on bioRxiv</a> in January 2026, a flow-matching model for de novo protein binder design paired with an in silico maturation step. Across seven mini-binder targets, they tested 30 designs each (210 total), expressed all 210, and reported a 36.2% sub-µM hit rate (76/210). Best KD per target reached 1 pM for PDGFR and VEGFA, 1.38 pM for IFNAR2, and 5.14 pM for PD-L1. Across eight VHH targets (240 designs), they report 81 total binders (33.8%) and 67 sub-µM binders (27.9%), with a best of 250 pM for CCL2. The headline claim is picomolar and nanomolar binders with no experimental affinity maturation.</p><p>The pipeline generates binder backbones by flow matching over residue frames, sequences them with ProteinMPNN or AbMPNN, packs side chains with FlowPacker, runs a maturation step that fixes interface anchor rotamers and regenerates from a partial flow state, and ranks candidates with AF3Score, a score-only adaptation of AlphaFold3. Code and checkpoint links are public. The repo implements the broad workflow, but the released examples do not reproduce the paper workflow out of the box.</p><p>The paper also has gaps. The picomolar values come from a BLI assay with a 300 s dissociation phase fit to a 1:1 model, and the tables report KD with no kon or koff. The four deepest mini-binder values come from Fc-fusion or homodimeric targets run as the analyte against a monovalent binder, a format that can produce avidity. The per-target headline is the single tightest design, not the affinity distribution. No binder was tested in a functional or neutralization assay. The epitope was never validated experimentally, no complex was solved, and the affinity was never confirmed by a second method. There is no specificity or polyreactivity panel for the mini-binders, and no SEC, DLS, or MALS on the mini-binder designs. The released ranking code also uses the composite score with the opposite sign from the methods section.</p><p>I’m going to walk through what they did, then go through the validation gaps I found.</p><h3>How It Works</h3><p>PPIFlow models binder design as a continuous flow over SE(3) residue frames. A pairformer reasons over interface contacts and Invariant Point Attention decodes the frame updates. Training uses a multi-stage curriculum across monomers, protein complexes, scFvs, and VHHs.</p><p>The in silico maturation stage runs after initial generation. The model identifies anchor rotamers on the interface that score below −5 Rosetta Energy Units, fixes them, perturbs the backbone to an intermediate flow state (t = 0.6), then regenerates the perturbed regions and redesigns the sequence around the fixed anchors.</p><p>For mini-binders, the methods say first-round ProteinMPNN used T = 0.2, with 8 vanilla sequences and 8 sequences biased toward bulky hydrophobic residues per backbone. Across those 16 sequences, Rosetta residue-wise interaction energies were used to merge the residues below −5 REU into an interface rotamer enriched design. The second round used partial flow at t = 0.6 and ProteinMPNN soluble redesign at T = 0.1.</p><p>For VHHs, the methods say they sampled 25,000 backbones per target, designed 8 CDR sequences per backbone with AbMPNN at T = 0.5, kept candidates with ipTM &gt; 0.2, and then ran the same rotamer-enrichment plus partial-flow idea. In the second round, 8 refined backbone conformations were generated and 4 sequences per refined structure were redesigned at T = 0.1. Second-round candidates were filtered at ipTM &gt; 0.5 and pTM &gt; 0.8, then validated with AlphaFold3 and DockQ.</p><p>Candidate ranking uses AF3Score, which retains the AlphaFold3 confidence head but skips full structure inference, giving about a 100-fold speedup. That speedup is what makes the generation funnel tractable. The paper then ranks final candidates by a composite score: AF3 ipTM × 100 − Rosetta interface score.</p><h3>Repository Check</h3><p>The public repo matches the paper at the pipeline-concept level. It has Stage 1 generation, sequence design, FlowPacker side-chain packing, AF3Score filtering, Rosetta interface analysis, fixed-position extraction, partial redesign, second-round scoring, AF3 refolding, DockQ, RosettaRelax, ranking, and report generation. It also provides links for four checkpoints: binder, antibody, nanobody, and monomer.</p><p>The clearest discrepancy is in the ranking code. The paper ranks candidates by AF3 ipTM × 100 − Interface Score. The repo RankStep (steps/rank_step.py) computes rank_score = interface_score + 100 × ipTM and sorts descending. The interface_score it reads is Rosetta’s get_interface_dG (steps/rosetta/relax_complex.py), which is negative for a favorable interface. The paper subtracts the interface score, so a more favorable (more negative) interface raises the composite score. The repo adds it, so a more favorable interface lowers the composite score. At equal ipTM, the released ranking pushes the less favorable interfaces toward the top. This is a sign inversion relative to the published formula, and it changes which designs rise.</p><p>The released configs are also demo-scale, not the paper runs. The binder example sets samples_per_target to 10, and the demo task disables both PPIFlowStep and the stage-1 MPNN step, so the binder example starts from provided structures rather than generating. The binder config that does exist sets base ProteinMPNN, 8 sequences, T = 0.1, and partial-flow start_t = 0.7, against the paper’s T = 0.2 with 8 vanilla plus 8 biased sequences and partial flow at t = 0.6. The nanobody example uses 10 samples, AbMPNN T = 0.1, filters at ipTM_A_C &gt; 0.3 and chain_A_ptm &gt; 0.7, and sets run_AF3_inference to false, against the paper’s 25,000 backbones, AbMPNN T = 0.5, round-1 ipTM &gt; 0.2, and round-2 ipTM &gt; 0.5 and pTM &gt; 0.8 with AF3 validation.</p><p>There is also a path issue. The example step configs point to absolute paths such as /tools/PPIFlow and /tools/flowpacker, and the README states that many paths in the example configs must be replaced with local paths, so those are placeholders by design. Install.sh is a separate problem: it sets the tools directory to tools and changes into it, while the repository ships a tool directory (singular). Install.sh will fail at that step unless the directory is renamed.</p><p>So the repo is an open implementation of the pipeline concept, not a turnkey reproduction of the paper’s 210 mini-binder and 240 VHH experiments.</p><h3>Experimental Results</h3><p>All affinities were measured by BLI on a Gator Bio instrument.</p><h4>Mini-binders, 30 designs per target, 210 total:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*89y5ajI1QuCeJKxEg99IZQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U6HFQF2T9Vm6St7uIl6pvA.png" /><figcaption><strong>PPIFlow mini-binder design and experimental validation across seven targets.</strong> (a) Highest-affinity de novo binder per target, with expression, hit rates, best affinities, and representative BLI sensorgrams. (b) Per-target hit rates. (c) Per-target affinity distributions (violin plots). (d, e) Interface side-chain packing for PDL1_26 after initial generation (d) and in silico affinity maturation (e). (f) AF3Score profiles for Round 1 versus Round 2 designs. (g, h) AF3Score pTM (g) and ipTM (h) for IFNAR2 across design strategies.</figcaption></figure><h4>VHHs, 30 designs per target, 240 total:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*czz011W7CS-TaXONsA9lFg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8cRVOFUQBBP-dxiL6l8UFA.png" /><figcaption><strong>PPIFlow VHH design and experimental validation across eight targets.</strong> (a) Highest-affinity de novo VHH per target, with expression, binding hit rates, best KD, and representative BLI sensorgrams. (b) Per-target hit rates. (c) Per-target affinity distributions (violin plots), with previous-best binders marked. (d, e) Interface side-chain packing for CCL2_21 after initial generation (d) and in silico affinity maturation (e), with AF3Score ipTM and interface scores. (f) AF3Score complex ipTM for Round 1 versus Round 2 designs across targets. (g, h) AF3Score VHH pTM (g) and complex ipTM (h) for CCL2 across design strategies.</figcaption></figure><p>Expression was 100% in both campaigns. The sub-µM hit rate was 36.2% for mini-binders and 27.9% for VHHs. The best-KD column is the single tightest design per target, not the median.</p><h3>Strengths</h3><p>Hit rate from a purely computational funnel. 36.2% of mini-binders bound below 1 µM across seven structurally diverse targets, with no wet-lab affinity maturation. For a generate-filter-test pipeline, that is a strong screen.</p><p>The VHH screen also worked across most targets. The paper reports binders for seven of eight VHH targets and 67 sub-µM designs out of 240 tested. The best VHH reached 250 pM against CCL2, and HNMT, PDGFR, and 1433E reached single-digit nanomolar best values.</p><p>The maturation ablations support the mechanism. Fixing anchor rotamers and running partial flow raised AF3Score pTM and ipTM and improved Rosetta interface scores relative to initial generation and refinement-only baselines. PDL1_26 went from −50.84 to −63.97 REU interface score and 0.76 to 0.85 ipTM; CCL2_21 went from 0.26 to 0.58 ipTM. The step does what the paper claims computationally.</p><p>AF3Score gives about a 100-fold scoring speedup over full AF3 inference, which is what makes large candidate funnels practical.</p><p>The open-source release is useful. The repo exposes the core pipeline, PPIFlow model code, example configs, and checkpoint links. It is not a turnkey reproduction of the paper’s experimental runs, but it is more than a static model card or a methods-only release.</p><h3>Gaps</h3><h4>Gap #1: The Best-KD Row Is One Design, Not the Distribution</h4><p>The per-target headline reports the single tightest binder, not the spread. For IL17A, Table S7 shows a broad distribution: one 747 pM design, several designs in the 10–100 nM range, and several weaker designs above 100 nM. The 36.2% sub-µM hit rate describes the screen. The best-KD row describes one outlier per target. The two numbers answer different questions, and the second one carries the picomolar claim.</p><h4>Gap #2: The 300 s Dissociation Window Cannot Resolve Picomolar Affinities</h4><p>The mini-binder assay uses a 120 s association phase, a 300 s dissociation phase, and a 1:1 fit. Take KD = 1 pM, the value reported for PDGFR and VEGFA. With a kon of 1e6 M⁻¹s⁻¹, koff is 1e−6 s⁻¹, so the complex loses about 0.03% of signal across the 300 s window. With a kon of 1e5 M⁻¹s⁻¹, it loses 0.003%. A 1:1 model cannot reliably fit an off-rate from a trace that is flat inside baseline noise, so the reported KD can become dominated by the lower resolvable koff range or fitting bounds. Two targets returning 1 pM is consistent with a fit approaching that floor.</p><p>To see even 5% dissociation across 300 s, koff has to be near 1.7e−4 s⁻¹, which paired with a typical kon is a KD in the high-picomolar to low-nanomolar range. That is the floor this assay resolves. The tables report KD only, with no kon and no koff, so the one number that would settle this is not shown.</p><h4>Gap #3: The Deepest Affinities Come From Bivalent Analytes Against Monovalent Binders</h4><p>The four tightest mini-binder values come from multivalent or dimeric targets run as analyte over a nominally monovalent, surface-immobilized His-tagged binder: PDGFR (Fc fusion, 1 pM), VEGF165 (native dimeric ligand, 1 pM), IFNAR2 (Fc fusion, 1.38 pM), and PD-L1 (Fc fusion, 5.14 pM). The methods confirm this orientation: for TrkA, IFNAR2, IL-7RA, PD-L1, PDGFR, and VEGF-A, the His-tagged binder was immobilized on the sensor and the target was the analyte. A bivalent analyte can bridge two surface-bound binders. The second arm rebinds before the complex fully releases, which suppresses the apparent off-rate far below the monovalent value. A 1:1 fit reads that slow apparent koff as ultra-high affinity. The reported numbers are apparent KDs for a bivalent interaction; the intrinsic monovalent KDs are not reported.</p><p>The orientation that avoids bridging gives a different answer. IL17A was run with the monovalent binder as the analyte over immobilized antigen, and the best design came in at 747 pM. IL7RA, an Fc fusion as analyte, gave 316 pM. TrkA, an Fc fusion too, stayed at 74.1 nM, because only 2 of 30 designs bound at all and avidity cannot rescue an interface that barely forms. The deepest affinities are concentrated among the bivalent-analyte targets, not the monovalent-analyte measurements.</p><h4>Gap #4: “Sub-Nanomolar VHHs Across Multiple Targets” Overstates the VHH Data</h4><p>The VHH bests are mostly nanomolar: CCL2 250 pM, HNMT 2.5 nM, PDGFR 4.09 nM, 1433E 6.47 nM, then 131 to 276 nM for BHRF1, S100A4, and EFNA1. IL13 produced no binders.</p><p>The abstract claims sub-nanomolar VHH binders across multiple targets, and the Figure 4 caption names CCL2, PDGFR, and 1433E as sub-nanomolar. In the same figure, CCL2 is 250 pM, but PDGFR is 4.09 nM and 1433E is 6.47 nM. One of the three is sub-nanomolar.</p><h4>Gap #5: No Orthogonal Affinity Confirmation</h4><p>All affinity measurements come from one platform: BLI on a Gator Bio instrument. No picomolar binder was confirmed by SPR, KinExA, equilibrium binding, competition titration, or another method that can independently test the affinity range.</p><p>This is the part that carries the 1 to 5 pM claims. A 300 s BLI dissociation trace with a bivalent analyte can return an apparent KD set by assay geometry and fitting limits. Orthogonal confirmation would distinguish a picomolar binder from a koff-limited BLI artifact.</p><h4>Gap #6: No Functional Data</h4><p>No neutralization or cell-based assay shows any binder blocks its target’s biological interaction. Binding to PD-L1, VEGFA, or IL17A is reported as KD, not as IC50 against the cognate interaction (PD-L1/PD-1, VEGFA/VEGFR, IL17A signaling). For targets defined by a function, binding without a blocking readout leaves the therapeutic claim untested.</p><h4>Gap #7: No Experimental Epitope Validation</h4><p>PPIFlow is described as targeting predefined epitopes through hotspots, and the design inputs list hotspot residues for each target. But the experimental data only show target binding. There is no competition experiment, alanine scanning, mutagenesis, HDX, or structure showing that the binders engage the intended epitope.</p><p>This is a major gap for an epitope-guided design method. A binder can hit the target and still bind somewhere else. Without epitope validation, the paper supports target binding, not successful epitope targeting.</p><h4>Gap #8: No Experimental Structures</h4><p>The paper shows AF3 models of the binder-target complexes, but no experimental structure of any complex. No complex was solved by cryo-EM, X-ray crystallography, or NMR.</p><p>That leaves two questions unresolved: whether the binder binds in the designed pose, and whether the predicted interface geometry exists experimentally. For a paper claiming direct computational design of picomolar binders, even one structure would have settled the binding mode.</p><h4>Gap #9: No Specificity or Polyreactivity Panel</h4><p>There is no off-target or polyreactivity testing for the mini-binders. The Figure S7 specificity note refers to multi-point kinetic titrations, not cross-reactivity against unrelated proteins. A de novo interface has no selection pressure against non-specific binding, so polyreactivity has to be measured after the fact. Standard options are baculovirus particle ELISA, PSR, heparin, insulin, DNA, or a panel of unrelated antigens. None were run.</p><p>The same point applies to VHH specificity. The paper shows target binding kinetics, but not a panel demonstrating that the designed binders avoid unrelated proteins.</p><h4>Gap #10: Limited Oligomeric-State and Developability Data</h4><p>For mini-binders, there is no SEC, DLS, or MALS on the designs. Aggregated or multimeric binder on the sensor can produce the same slow apparent off-rate as avidity, so the biophysical state of each design is part of the affinity question. A basic SEC and DLS characterization of the confirmed mini-binders would have separated a monomeric high-affinity binder from an aggregation artifact.</p><p>For VHHs, the methods mention SEC-HPLC during purification, but the paper does not report per-design SEC traces, oligomeric-state distributions, thermal stability, or aggregation behavior for the confirmed binders. That is less severe than the mini-binder case, but still incomplete for a therapeutic framing.</p><h4>Gap #11: The Repo Does Not Reproduce the Paper Workflow Exactly</h4><p>The repo implements the broad design pipeline, but the released configs are not the paper configs, and the ranking code does not match the paper formula.</p><p>The strongest mismatch is the ranking sign described above: RankStep computes interface_score + 100 × ipTM, while the paper specifies 100 × ipTM − Interface Score, and the Rosetta interface score is negative for favorable binders. The mini-binder example uses 8 ProteinMPNN sequences at T = 0.1 instead of the paper’s 8 vanilla plus 8 biased sequences at T = 0.2, and partial-flow start_t = 0.7 instead of t = 0.6. The VHH example uses 10 samples instead of 25,000, AbMPNN T = 0.1 instead of T = 0.5, looser AF3Score thresholds, and disables AF3 inference in the refold step. Install.sh changes into a tools directory while the repo ships a tool directory, so it fails at that step unless renamed.</p><p>This does not invalidate the paper. It changes how the open-source release should be described. The repo is an implementation of the pipeline concept, not an exact reproducibility package for the reported experiments.</p><h3>Conclusion</h3><p>PPIFlow generates expressed, target-binding proteins at a 36.2% sub-µM mini-binder rate from a purely computational funnel, and the maturation ablations show the partial-flow step improves interface metrics as described. The VHH campaign also produced binders for seven of eight targets, including a 250 pM CCL2 design. The open-source release is always commendable and appreciated.</p><p>But the strongest number in the paper is still the screen-level hit rate, not the single-design picomolar KD values. The picomolar values read as apparent affinities from a 300 s, koff-limited BLI assay, often in a bivalent-analyte format. The experiments that would test them are missing: the kinetic rate constants, an orthogonal affinity method, a functional assay, an experimental structure, and epitope mapping.</p><p>Here’s what would make this work stronger: report kon and koff for every picomolar binder, not just KD. Repeat the deep-pM measurements with monovalent assay formats and longer dissociation windows. Confirm at least one picomolar value with an orthogonal method such as KinExA or equilibrium competition. Run a functional IC50 against the cognate interaction for the therapeutic targets. Map epitopes with competition, mutagenesis, or a structure. Run a polyreactivity panel. Add SEC, DLS, or SEC-MALS on the confirmed mini-binders to rule out aggregation. Release the exact paper configs and the ranking formula used to select the tested designs.</p><p>Until then, the sub-µM hit rate is the number the data supports, and the picomolar values should be treated as apparent BLI affinities rather than intrinsic monovalent KDs.</p><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><p><em>Views expressed are my own and do not represent my employer.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a0336bf526a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Stress-Tested the AWS-MSK Nanobody Paper. These Are the Gaps.]]></title>
            <link>https://medium.com/@enginyapici/i-stress-tested-the-aws-msk-nanobody-paper-these-are-the-gaps-37188c530465?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/37188c530465</guid>
            <category><![CDATA[drug-discovery]]></category>
            <category><![CDATA[antibody-engineering]]></category>
            <category><![CDATA[protein-design]]></category>
            <category><![CDATA[antibody-design]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 21 Apr 2026 06:23:33 GMT</pubDate>
            <atom:updated>2026-04-24T07:13:34.084Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Stress-Tested the AWS–MSK Nanobody Paper. These Are the Gaps.</h3><p><em>Technical analysis of an agent-guided de novo nanobody design campaign.</em></p><p>AWS Life Sciences and Memorial Sloan Kettering released a <a href="https://www.biorxiv.org/content/10.64898/2026.04.13.717816v1">paper</a> on <em>de novo</em> nanobody design against a novel Desmoplastic Small Round Cell Tumor (DSRCT) target. The campaign used no prior antibody information and had no experimental structure to work from. 288,000 computational designs, 100,000 of them screened by yeast display, 116 advanced to SPR, 46 confirmed binders with Rmax ≥ 30 RU.</p><p>The top binder: PRJ266_044, KD = 0.66 nM.</p><p>Median KD across the 46 confirmed binders: 31.7 nM. Range: 0.66 to 305 nM.</p><p>This is one of the first published <em>de novo</em> antibody campaigns against a target with neither a wet-lab structure nor any prior antibody-antigen interaction data. The workflow combines an LLM-driven hotspot recommendation agent, three generative models (RFantibody, IgGM, mBER), five predicted antigen conformations, three nanobody frameworks, and Pareto-based candidate selection.</p><p>The paper also has gaps. We don’t know the identity of the target protein or how similar it is to other proteins with deposited structures and antibodies. Framework diversity collapsed to one scaffold at the SPR stage. Epitope engagement was not tested experimentally. Developability was not characterized. Most of the sub-nanomolar KDs come from low-signal SPR traces the authors flag as provisional. One of the three generative methods produced no candidates above the reliability threshold. The three sensorgrams in the supplementary show dissociation curves that look like experimental artifacts.</p><p><em>Before going further, I want to highlight one thing about the paper I found very unusual: target name is not disclosed. The paper describes the selection process, but the protein itself is redacted throughout. This is a new de novo antibody design workflow being published with a big claim that it works against a target lacking experimental structure and any prior antibody data. Without target identity, we cannot check whether the target has 30% identity to an obscure orphan protein or 90% identity to something with 10 deposited antibody co-crystal structures. The folding models used here (AlphaFold2, Boltz-2, Chai-1, IntelliFold) are trained on the PDB, so a target with close structural relatives in the training data is a fundamentally easier problem than a target without them. The same goes for IEDB, which the hotspot agent queries directly. If the target has close homologs with annotated epitopes, the agent’s recommendations rest on prior antibody data by proxy. None of this can be assessed because the target is undisclosed.</em></p><h3>How It Works</h3><p>The pipeline has five stages:</p><p><strong>1. Target structure prediction and hotspot selection</strong><br>The target antigen structure was predicted independently by five methods: AlphaFold2, Boltz-2 (with and without potential guidance), Chai-1, and IntelliFold.</p><p>A hotspot recommendation agent was built via Claude Sonnet 4 orchestrating various bioinformatics tools:<br>- IEDB epitope search<br>- PFAM domain annotation<br>- Solvent-accessible surface area<br>- Secondary structure<br>- Hydrophobicity profiling</p><p>The agent was validated on a SAbDab holdout set of 76 immunologically relevant targets and achieved ~80% top-5 accuracy for epitope recovery. <em>That validation used additional internal tools (Unique Regions Analyzer, and Interface Analyzer which was not used by the agent for this target) not listed above and not independently documented, so the number reflects the full agent rather than the five external tools alone.</em></p><p><strong>2. De novo design<br></strong>Three generative methods with distinct principles:</p><ul><li>RFantibody (diffusion-based backbone generation)</li><li>IgGM (joint sequence-structure diffusion)</li><li>mBER (backpropagation through structure prediction)</li></ul><p>Each method generated 96,000 designs, varying across 3 frameworks, 5 predicted antigen structures, 8 hotspots, and CDR H3 lengths from 4 to 13 amino acids. Total library: 288,000.</p><p><strong>3. Multi-property scoring<br></strong>NanobodyBuilder2 pLDDT for monomer folding, Boltz-2 co-folding for ipTM, complex ipLDDT, and minimum CDR-antigen distance, MochiBind (an internally developed ESM-2-based sequence-level binding affinity predictor, currently under peer review), and liability checks adapted from the LAP methodology.</p><p><strong>4. Candidate selection<br></strong>A multi-objective Pareto optimization agent filtered 288,000 designs down to 100,000 using five properties (see the table below).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9Tw3HTHQcNrrhGVzBh1COw.png" /></figure><p><strong>5. Experimental screening<br></strong>Yeast surface display with two rounds of FACS enrichment. 116 candidates were advanced to SPR based on mean fluorescence intensity at Round 2. All 116 expressed as VHH-Fc fusions in HEK293 cells. Median yield: 368 µg per 2 × 1 mL culture. 46 produced Rmax ≥ 30 RU (Carterra LSA) with measurable kinetics. Specificity was confirmed against an irrelevant control (TfR1) for all 116 candidates.</p><h3>Experimental Results</h3><p>The workflow produced 46 SPR-confirmed binders with Rmax ≥ 30 RU and measurable kinetics. KD values span 0.66 nM to 305 nM with a median of 31.7 nM. The top candidate (PRJ266_044) is reported at KD = 0.66 nM. The library expressed at 90.6% in yeast. All 116 SPR candidates expressed as VHH-Fc fusions in HEK293 at a median yield of 368 µg at 2 mL scale, with 97% above 200 µg. All 116 passed a specificity check against the off-target control TfR1. Binders were recovered from designs conditioned on all 8 hotspot regions.</p><h3>Gaps</h3><h4>Gap #1: Target Identity Is Redacted</h4><p>The target is described through its selection criteria (differential expression in DSRCT, downstream of EWSR1::WT1, cell-surface localized, minimal expression in normal GTEx tissues, the most differentially upregulated gene remaining after filtering). The protein itself is redacted throughout the main text, the figures, and the supplementary info.</p><p>The folding models used here (AlphaFold2, Boltz-2, Chai-1, IntelliFold) train on the PDB. A target with close structural relatives in the training data is a fundamentally easier problem than a target without them. The hotspot agent also queries IEDB directly, which returns alignment hits from homologs with annotated epitopes. If the target has close homologs with known epitopes, the agent’s recommendations rely on prior antibody data by proxy.</p><p>The paper’s specificity control is TfR1. Whether TfR1 is an appropriate negative control depends on the target. For a claudin family member, TfR1 is fine. For a protein with structural similarity to TfR1, it’s too easy.</p><h4>Gap #2: No Polyreactivity or Non-Specific Binding Controls</h4><p>The paper’s specificity testing consisted of one off-target protein (TfR1). No binding was detected against TfR1 for any of the 116 candidates.</p><p>A single negative control protein does not rule out polyreactivity. Polyreactive antibodies bind many unrelated targets through non-specific surface interactions. Standard practice is to test against baculovirus particle (BVP) via ELISA at a minimum. Additional options include PSR assay, dsDNA, ssDNA, insulin, LPS, cardiolipin, ovalbumin, lysozyme-binding tests. The paper reports none of these.</p><h4>Gap #3: The SPR Data Does Not Support the Reported Affinities</h4><p>The paper reports 16 candidates with sub-nanomolar KD values. Two cleared the Rmax ≥ 30 RU reliability threshold (PRJ266_044 and PRJ266_080), but the sub-nanomolar binder PRJ266_104 has low Rmax (12.1 RU). The other 14 have Rmax &lt; 30 RU. The paper flags these as provisional and lists the failure modes: low active concentration, partial binder immunoreactivity, reduced reliability of kinetic parameter estimation with low signals.</p><p>None of the three sensorgrams support a confident kinetic fit. PRJ266_044 and PRJ266_080 show biphasic dissociation with a plateau at the start. PRJ266_104 has a different artifact: a sudden signal increase at the beginning of the dissociation phase followed by a flat trace, which makes any fit koff-limited assuming binding occurred at all. The only thing these sensorgrams demonstrate is that there were experimental artifacts that need to be addressed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ca60SNgPGiGAPDaruQyaOw.png" /></figure><h4>Gap #4: All Binders Were Tested in Bivalent VHH-Fc Format</h4><p>The design models generated single-domain VHH sequences. The experimental validation was done with the VHHs expressed as VHH-Fc fusions in HEK293. A VHH-Fc has two VHH domains linked by the Fc hinge-CH2-CH3 and binds bivalently.</p><p>The reported KD values are apparent KDs for the bivalent format. Intrinsic monovalent KDs of the designed VHHs are not reported. Bivalent binding produces avidity effects that suppress apparent off-rates and amplify apparent affinity by orders of magnitude, depending on epitope geometry and valency accessibility.</p><p>The 0.66 nM KD for PRJ266_044 could reflect a high-affinity VHH, a moderate-affinity VHH with avidity amplification, or some combination.</p><h4>Gap #5: Framework Diversity Collapsed at the SPR Stage</h4><p>The design campaign used three VHH frameworks. Among the 46 binders with Rmax ≥ 30 RU:</p><ul><li>Framework A: 1 binder</li><li>Framework B: 45 binders</li><li>Framework C: 0 binders</li></ul><p>45 of 46 confirmed binders came from a single framework. Frameworks A and C were represented in the 288,000 initial designs and in the 100,000 Pareto-selected candidates (Figure S3 in the paper). The stage-by-stage attrition by framework between the 100,000 selected candidates and the 46 confirmed binders is not reported. The paper does not name the three frameworks.</p><h4>Gap #6: Epitope Specificity Was Not Experimentally Confirmed</h4><p>The hotspot agent proposed 8 distinct epitope regions. The design library was conditioned on those regions. Boltz-2 co-folding was used to check whether the designs dock near their conditioned hotspots. Designs conditioned on Hotspots F, G, and H show contact peaks at those hotspots (figure below). Boltz-2 was trained on the same PDB data as the design models and shares their biases, so agreement between them is expected even if the designs don’t bind where predicted. Only experimental mapping can confirm the binding sites.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h5llLcu_QZ8DgGbfN7LHFA.png" /><figcaption><strong>Paper Figure 4 (b-d).</strong> (b) Boltz-2 co-folding of F-conditioned and G-conditioned designs against the antigen. Both sets show contact peaks at Hotspot F and Hotspot G, along with substantial contacts outside the conditioned regions. (c) Aggregated CDR contact frequencies across all 46 SPR-confirmed binders (Rmax ≥ 30 RU). Contacts defined as any CDR heavy atom within 7 Å of any antigen heavy atom. Shaded bands mark the 8 agent-defined hotspots. The highest contact peak sits at Hotspot B, with additional peaks at F, G, and H. (d) Per-design KD against contact position. Each binder contributes multiple dots (one per contacted residue). Violin plots show KD distributions for designs whose contacts fall within ±10 residues of each hotspot.</figcaption></figure><p>The paper states this directly: <em>“SPR confirms target binding but does not resolve which epitope is engaged; experimental epitope mapping would be needed to determine the true binding sites.”</em></p><p>This is the same gap that has appeared in several other recent de novo antibody papers. With 46 SPR-confirmed binders in hand, competitive binning on 20–25 representatives would be a good addition.</p><h4>Gap #7: Sensorgrams Are Shown for Only 3 of 46 Confirmed Binders</h4><p>The paper reports 46 binders with Rmax ≥ 30 RU and measurable kinetics. Figure S2 shows sensorgrams for 3 of them. The remaining 43 sensorgrams are not shown in the paper or supplementary.</p><h4>Gap #8: In Silico Filtering Metrics Do Not Separate Binders From Non-Binders</h4><p>Figure S7 compares five computational metrics between the 46 SPR-confirmed binders and the 41 non-binders from the 116 candidates characterized by SPR. The Mann-Whitney p-values:</p><ul><li>MochiBind predicted binding affinity: p = 0.0227</li><li>iPTM: p = 0.09</li><li>NanobodyBuilder2 average pLDDT: p = 0.09</li><li>Complex ipLDDT: p = 0.19</li><li>Minimum CDR-antigen distance: p = 0.75</li></ul><p>Only MochiBind crosses the 0.05 threshold.</p><p>Four of these five metrics were used as Pareto optimization objectives to filter 288,000 designs down to 100,000. The one metric that does discriminate binders from non-binders (MochiBind) is the sequence-based predictor whose manuscript is under peer review. The paper does not report how much of the 288,000 → 100,000 reduction was driven by MochiBind versus the four structure-based metrics.</p><p>The paper acknowledges the limited correlation: “we observed little correlation between most in silico folding metrics (pLDDT, ipTM) and experimental binding outcomes in the SPR data.”</p><h4>Gap #9: No Developability Characterization</h4><p>The 46 confirmed binders have affinity data and one specificity data point (TfR1). The paper reports no:</p><ul><li>Thermal stability measurements (DSF, DSC, nano-DSF)</li><li>Aggregation propensity (SEC, DLS, AC-SINS)</li><li>Self-interaction measurements (CIC, HIC)</li><li>Expression yield characterization beyond the 2 mL culture scale</li><li>Cell-based binding against target-expressing cells</li></ul><p>The paper flags developability as planned future work in the DBTL framework (Figure S8).</p><h4>Gap #10: The 39.7% Hit Rate Lacks a Baseline</h4><p>The paper reports 46 of 116 SPR-characterized candidates as confirmed binders (39.7%). Two pieces of context are missing:</p><ul><li>First, no negative control library. Random CDRs on Framework B, run through the same yeast display, FACS, and SPR pipeline, would need to be tested to set a baseline.</li><li>Second, 39.7% is the SPR success rate on candidates already pre-enriched through yeast display and two rounds of FACS. The library-level hit rate is different. Of the 100,000 designs that went into yeast display, what fraction passed Round 2 FACS is not reported. The NGS sequencing of binding-positive populations mentioned in the paper could provide this number.</li></ul><h3>Conclusion</h3><p>The AWS/MSK workflow produced 46 SPR-confirmed binders against a DSRCT target using an integrated agent-guided <em>de novo</em> design pipeline. The agent framework is transparently disclosed. The computational exploration covered 288,000 designs, 100,000 screened, 116 to SPR.</p><p>Here’s what would make this work stronger: Publish a homology analysis with the target’s sequence and structural similarity to proteins in PDB and SAbDab. No target name required. The question is whether close homologs with deposited structures or antibodies exist. Repeat the SPR experiments for top binders with fresh chip, buffer, and analyte. Test the VHHs in monovalent format. Show sensorgrams for all 46 confirmed binders. Run a polyreactivity panel. Run a negative control library of random CDRs on Framework B through the same funnel. Run experimental epitope mapping on a representative subset.</p><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><p><em>Views expressed are my own and do not represent my employer.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37188c530465" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Germinal: Open-Source Nanobody Design With Epitope Evidence and Some Unresolved Gaps]]></title>
            <link>https://medium.com/@enginyapici/germinal-open-source-nanobody-design-with-epitope-evidence-and-some-unresolved-gaps-2b6dfac3140c?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b6dfac3140c</guid>
            <category><![CDATA[protein-engineering]]></category>
            <category><![CDATA[drug-discovery]]></category>
            <category><![CDATA[biotechnology]]></category>
            <category><![CDATA[pharma]]></category>
            <category><![CDATA[protein-design]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 24 Feb 2026 07:57:15 GMT</pubDate>
            <atom:updated>2026-02-24T08:38:40.025Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Stanford and Arc Institute’s pipeline designs nanobodies against specified epitopes, testing only 43–101 candidates per target.</em></p><p>A Stanford/Arc Institute team released <a href="https://www.biorxiv.org/content/10.1101/2025.09.19.677421v2">Germinal</a> in Sep 2025, an open-source generative framework for de novo nanobody design. The approach combines <a href="https://www.biorxiv.org/content/10.1101/2021.10.04.463034v2">AlphaFold-Multimer</a> (AF-M) for structural confidence with <a href="https://github.com/Graylab/IgLM">IgLM</a>, an antibody-specific language model, for sequence naturalness. CDRs are designed from scratch on a fixed framework, filtered with <a href="https://github.com/google-deepmind/alphafold3">AF3</a> and <a href="https://www.pyrosetta.org/">PyRosetta</a>-derived biophysical and biochemical scores, and validated experimentally.</p><p>The results across four diverse targets: 4–22% BLI-verified success rates from libraries of just 43–101 nanobodies per antigen. Best affinities reached 140 nM (BHRF1), 170 nM (PD-L1), 190 nM (IL20), and 560 nM (IL3). All nanomolar. All from small design pools.</p><p>I’m going to walk through what they did, then go through the validation gaps I found.</p><h4>How the Model Works</h4><p>Germinal works in three stages: design, sequence optimization, and filtering.</p><p>In the design stage, gradients from AF-M and IgLM are merged at each iteration to co-optimize structure confidence and antibody-likeness. Three custom loss functions steer CDRs away from alpha-helices and beta-strands (which AF-M otherwise favors) and enforce that binding contacts occur through CDR residues rather than the framework. The design progresses through logit optimization, softmax annealing, and semi-greedy discrete selection (see appendix for more info). Each design iteration takes 2–8 minutes on an H100 GPU (per their repo <a href="https://github.com/SantiagoMille/germinal">README</a>), and full campaigns require 200–500 H100-GPU hours per target.</p><p>In the sequence optimization stage, <a href="https://arxiv.org/abs/2310.19513">AbMPNN</a> redesigns non-contact CDR residues to improve stability while preserving the predicted binding interface.</p><p>In the filtering stage, AF3 provides an independent structural assessment (different from the AF-M used during design), and PyRosetta scores provide biophysical metrics. Only designs passing stringent thresholds on ipTM, pLDDT, PAE, CDR interface percentage, shape complementarity, hydrogen bonds, and several other criteria advance to experimental testing.</p><p>Using AF3 as a separate validator from the AF-M used during optimization reduces the risk of the pipeline finding sequences that game a single model’s confidence metrics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JRvH1uMlAUhC56qZskMBDg.png" /><figcaption><strong>Germinal pipeline overview and experimental results.</strong> (A) The pipeline takes a target structure and antibody framework as input, designs CDRs by merging AF-M and IgLM gradients through three phases (logits, softmax with temperature annealing, semi-greedy selection), then optimizes non-contact CDR residues with AbMPNN and filters with AF3. (B) AF-M and IgLM define separate optima in sequence space; joint optimization steers toward sequences that score well on both objectives. (C) AF3-predicted structures of the best binders for each target, with CDRs in pink and target binding domains in blue. Best KD values by BLI are annotated. (D) Screening results per target. Of 43–101 designs tested per library, 4–22% were verified as binders by BLI. “Max. Id.” is the highest pairwise CDR sequence identity between any design and the PDB, confirming the designs are novel.</figcaption></figure><p>The funnel from computation to validated binders looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c4KZM7udU9kYYKHFJtdxRA.png" /></figure><p>The total number of trajectories attempted before hallucination filtering isn’t reported. The <a href="https://github.com/SantiagoMille/germinal">README</a> in the GitHub repo says they typically run “until there are around 1,000 passing designs,” which at ~1 pass per GPU hour implies thousands of total attempts per target. The dropoff from hallucination to experimental selection is steep: only the top 3–7% of passing designs were chosen for testing based on the metrics listed in the table below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dmuLeA0OyuD_YEc_oYI9eQ.png" /></figure><h4>Target Diversity</h4><p>They tested against four <strong>soluble</strong> proteins:</p><ul><li><strong>PD-L1: </strong>101 designs tested, 7 BLI-verified binders (7%), best KD = 170 nM.</li><li><strong>IL3: </strong>a cytokine with no previously reported <em>de novo</em> binders. 46 designs, 2 BLI-verified binders (4%), best KD = 560 nM.</li><li><strong>IL20: </strong>another cytokine without prior de novo binders. 43 designs, 4 BLI-verified binders (9%), best KD = 190 nM (KD1 from a 2:1 heterogeneous ligand fit).</li><li><strong>BHRF1: </strong>an Epstein-Barr virus Bcl-2-like protein. 52 designs, 11 BLI-verified binders (22%), best KD = 140 nM.</li></ul><p>All input target structures were AF3 predictions rather than crystal structures, which demonstrates the pipeline can work without experimentally solved target structures.</p><h4>Epitope Targeting Evidence</h4><p>For PD-L1, they introduced alanine substitutions at three predicted interface residues (I54, Y56, E58). Each substitution reduced binding by &gt;10-fold for some designs and had little effect on others. This pattern makes sense if different nanobodies contact different residues within the epitope. A nanobody that relies heavily on Y56 for binding loses affinity when Y56 is mutated to alanine, but a nanobody that primarily contacts I54 is unaffected by that same mutation. If the mutations were simply destabilizing PD-L1 as a whole, all nanobodies would lose binding equally. This differential sensitivity pattern is consistent with the designed binders contacting the intended epitope rather than binding somewhere else entirely.</p><p>However, the alanine scan was only performed on PD-L1. There’s no epitope validation for IL3, IL20, or BHRF1. Given that IL3 and IL20 are new targets where no prior binder data exists, epitope mapping on at least one additional target would strengthen the claim that Germinal’s epitope targeting generalizes.</p><h4>Screening Assay and Controls</h4><p>The experimental workflow uses a split-luciferase (<a href="https://www.promega.com/products/protein-interactions/live-cell-protein-interactions/nanobit-ppi-starter-systems/">NanoBiT</a>) assay as the initial screen. Binder-LgBiT fusions expressed in HEK293 cells are tested for binding by adding antigen-SmBiT fusions. They included both positive controls (known binders to the same target, when available) and negative controls (binders against unrelated antigens).</p><p>The assay shows standard curves with R² &gt; 0.97 for both expression (HiBiT) and binding (SmBiT).</p><p>The dropout between luciferase hits and BLI-verified binders is substantial. For PD-L1: 25 candidates passed luciferase filters, but only 7 showed detectable binding on BLI. For BHRF1: 20 passed luciferase, 11 confirmed on BLI.</p><h4>Strengths</h4><ul><li>Full open-source release with pipeline code, hyperparameter configs for PD-L1 and IL3, all designed sequences, and Docker/Singularity containerization.</li><li>Alanine scanning on PD-L1 provides good epitope targeting evidence.</li><li>Split-luciferase screening assay is practical, well-characterized with standard curves, and accessible to labs without specialized high-throughput equipment.</li><li>Both positive and negative controls included in each screening plate.</li><li>Luciferase-to-BLI dropout rates transparently reported.</li></ul><h3>Gaps</h3><h4>Gap #1: No Polyreactivity or Polyspecificity Testing</h4><p>The paper’s specificity testing is limited to a single off-target protein per screen (IL2 for PD-L1 binders; PD-L1 for the other three targets). For the BLI specificity check (Figure S12), they tested only two binders per target against one off-target antigen at a single concentration (2000 nM).</p><p>Testing against one off-target protein doesn’t tell you whether a binder is polyreactive. Polyreactive antibodies bind many unrelated targets through non-specific hydrophobic or electrostatic interactions. You can pass a single-antigen specificity check and still be polyreactive. The standard assays for this are baculovirus particle (BVP) ELISA, polyspecificity reagent (PSR) binding, heparin chromatography, or at minimum a panel of unrelated antigens. None of these were performed.</p><p>The authors do acknowledge this directly: “more comprehensive poly-specificity measurements will be required to rigorously confirm antigen specificity.”</p><p><strong>How nature handles polyreactivity:</strong> During B cell development, antibodies pass through multiple tolerance checkpoints that actively remove polyreactive clones from the repertoire. In the bone marrow, immature B cells expressing polyreactive BCRs undergo receptor editing or clonal deletion. Those that escape face additional selection in the periphery, where chronically self-reactive cells are anergized or excluded from germinal centers. <em>De novo</em> designed CDRs skip all of these checkpoints. There is no selection pressure against non-specific hydrophobic or electrostatic interactions, which means polyreactivity has to be assessed experimentally after the fact.</p><h4>Gap #2: Affinities Are 10–100x Weaker Than Therapeutic Requirements</h4><p>Best affinities range from 140 nM to 560 nM across the four targets. Therapeutic antibodies typically need single-digit nanomolar or better to be clinically viable. These affinities are starting points that would require extensive affinity maturation.</p><p>The paper frames these as strong results (“nanomolar binding affinities” in the abstract), but context is important here. The PD-L1 positive control (KN035) has a KD of 3 nM. The best Germinal PD-L1 binder is 57-fold weaker at 170 nM. For IL3, the best binder is 560 nM, which is closer to micromolar territory.</p><p>The authors do note that protein language models could enable affinity maturation as a future step.</p><h4>Gap #3: No Experimental Developability Data</h4><p>The paper presents computational developability metrics (Therapeutic Nanobody Profiler - TNP scores, OASis humanness scores, surface hydrophobicity, charge distribution) and shows that IgLM guidance improves these metrics. But there’s no experimental measurement of any developability parameter.</p><p>At a minimum, I’d want to see the following to assess whether these designs are viable starting points, with extended assays (marked with *) as bonuses:</p><ul><li>Thermal stability by DSF or DSC</li><li>Analytical SEC for aggregation and oligomeric state</li><li>Expression yields across the design set</li><li>Polyreactivity panel (BVP, PSR, or heparin chromatography)</li><li>Self-interaction as viscosity proxy (e.g., via AC-SINS)*</li><li>Polydispersity with DLS*</li></ul><p>The framework choice (hNbBCII) is well-characterized and likely does confer stability, but designing new CDRs onto a stable framework can still introduce aggregation-prone motifs, affect thermal stability, or create developability liabilities.</p><p>Even a basic DSF + DLS screen on the 24 BLI-verified binders would have provided useful data. The purified proteins were already in hand. They could have also run <a href="https://www.paiabio.com/">PAIA</a>’s plate-based developability kits to get some good assessments quickly with little material:</p><ul><li>Hydrophobicity (HIC assay)</li><li>Polyreactivity against heparin</li><li>Polyreactivity against ovalbumin</li></ul><h4>Gap #4: IL20 Kinetics Require a 2:1 Binding Model, and BLI Loading Levels Are Not Reported</h4><p>The IL20 sensorgrams required fitting to a 2:1 heterogeneous ligand model rather than the standard 1:1 kinetics used for the other targets. The paper explains that 1:1 fits showed systematic residuals during dissociation, and the 2:1 model minimized residuals.</p><p>Biphasic kinetics on BLI can arise from several sources. Analyte aggregation at the sensor surface has been shown to produce biphasic association and dissociation that worsens with longer association times and higher analyte concentrations (<a href="https://pubs.acs.org/doi/10.1021/acsomega.5c03942">Sherer and Cho, ACS Omega 2025</a>). High ligand loading density can introduce mass transport limitation and analyte rebinding artifacts (<a href="https://www.sciencedirect.com/science/article/pii/S0003269717303342">Kamat &amp; Rafique, Anal. Biochem. 2017</a>).</p><p>Conformational heterogeneity of the immobilized ligand or the analyte is another possibility. The nanobody analytes are monovalent, so bivalent binding can be ruled out. But the paper doesn’t investigate which of the remaining causes is responsible.</p><p>The paper never reports actual sensor loading levels for any target. The methods describe loading biotinylated target at 200 nM “for 300 s or until a 1 nm BLI shift was reached,” but the achieved loading levels are not disclosed. High ligand density on streptavidin sensors is one of the most common causes of biphasic kinetics in BLI, because it creates heterogeneous binding populations and can introduce mass transport limitation and analyte rebinding artifacts. Without knowing the loading levels, we can’t assess whether the IL20 biphasic behavior (and potentially the IL3 poor fits) reflects an assay artifact or something intrinsic to the interaction.</p><p>If they had tested at lower loading densities and the biphasic behavior went away, that points to an assay artifact. If it persisted, the two kinetic populations are likely intrinsic to the interaction. Flipping the assay orientation (immobilizing the nanobody, flowing IL20 as analyte) would also help. Neither experiment was done.</p><h4>Gap #5: IL3 BLI Fits Below Quality Threshold</h4><p>For IL3, both binders showed R² = 0.93 on the kinetic fits. The paper acknowledges this falls below their own quality threshold of R² &gt; 0.95 for the other targets. To compensate, they averaged KD values across replicate experiments and tested against both C-terminal and N-terminal AviTag versions of IL3.</p><p>Averaging replicates of a poor-fit model doesn’t improve the underlying fit quality. It reduces random error but doesn’t address systematic deviation between the model and the data. If the 1:1 binding model doesn’t describe the sensorgrams well, the extracted KD is unreliable regardless of how many times you repeat the experiment. The same loading-level concerns from IL20 apply here: without knowing the ligand density on the sensors, we can’t distinguish poor fits caused by the interaction itself from those caused by assay artifacts.</p><p>The reported IL3 KD values (560 nM and 2.5 µM) are approximate at best. The weaker of the two binders is already in the micromolar range.</p><h4>Gap #6: Configs in the Repo Don’t Fully Match the Paper</h4><p>The GitHub repo provides exact hyperparameter configs for PD-L1 and IL3, but not for IL20 or BHRF1. Anyone wanting to reproduce the IL20 or BHRF1 results would need to guess the parameters, and the README notes that “final sampling runs in the preprint all used slightly modified parameters” from the defaults.</p><p>There’s also a species conditioning detail that isn’t discussed in the paper. The PD-L1 config conditions IgLM on [CAMEL] species while IL3 uses [HUMAN]. Since the hNbBCII-FGLA framework originates from a camelid scaffold, [CAMEL] conditioning isn&#39;t unreasonable. But the paper doesn&#39;t mention that species conditioning was varied per target. The methods section gives &lt;HUMAN&gt; as the example species token, implying uniform conditioning.</p><h4>Gap #7: scFv Results Built on a Framework That Failed Grafting Controls</h4><p>The paper reports computational scFv design results (Figures S5, S6) with metric distributions comparable to nanobodies. None were tested experimentally.</p><p>The framework validation (Figure S1) shows why. They tested two scFv frameworks: trastuzumab and FW1.4. FW1.4 didn’t express. Trastuzumab expressed but lost all binding activity when grafted with known functional CDRs from F5111. They chose trastuzumab anyway and used it for all computational scFv designs.</p><p>The question is why include scFv results at all. They had no working scFv framework, they didn’t screen additional frameworks to find one that could support CDR grafting, and they didn’t test any of the computational designs. Computational metrics on a framework that can’t support known binders don’t add information.</p><h4>Gap #8: Expression Failures Not Analyzed</h4><p>Looking at Figure 1D, substantial fractions of designs failed to express: 23/101 for PD-L1 (23%), 10/46 for IL3 (22%), 13/43 for IL20 (30%), and 21/52 for BHRF1 (40%).</p><p>The paper’s differential analysis (Figure S7A) identified a few metrics correlated with expression (CDR SAP score, pLDDT), but doesn’t examine why expression failure varies so much across targets.</p><p>Since all designs use the same framework, expression failures should be driven by CDR properties. A systematic analysis of what distinguishes expressed from non-expressed designs (CDR hydrophobicity, aggregation-prone motifs, unusual amino acid compositions) would improve future design cycles.</p><h4><strong>Gap #9: No Audit of False Negatives from the Luciferase Screen</strong></h4><p>The luciferase filter removes a large fraction of designs before they ever reach BLI. For PD-L1, only 25 out of 78 expressing designs were sent to BLI. For BHRF1, 20 out of 31. The reported success rates (4–22%) are calculated from total designs tested per target, but the luciferase triage determines which of those designs get a chance at BLI confirmation. The authors themselves note the assay reflects “apparent association” and is influenced by transfection efficiency, cell debris, and other artifacts.</p><p>Taking 10–20 luciferase-failing designs per target and running them on BLI anyway would quantify the false negative rate. If even a few show binding, the luciferase filter is discarding functional binders and the conditional success rate (28–55% among designs that reached BLI) better reflects the pipeline’s design capability.</p><h4><strong>Gap #10: No Experimental Ablation of Pipeline Components</strong></h4><p>Germinal introduces paratope loss, alpha-helix loss, beta-strand loss, IgLM guidance, AbMPNN CDR redesign, and three different gradient merging strategies (weighted sum, PCGrad, MGDA). The paper shows computational effects of some of these (IgLM guidance improves predicted developability scores, secondary structure losses shift CDR conformations), but none of these ablations are connected to experimental outcomes.</p><p>Running even a small ablation on one target (e.g., designs with vs without IgLM guidance, tested through to BLI) would show whether the computational improvements translate to experimental differences. As it stands, some of these components could be computationally measurable but experimentally irrelevant.</p><h4><strong>Gap #11: Target Truncation and Presentation Format Not Validated</strong></h4><p>PD-L1 was truncated to the V-like domain (residues 19–127) for computational efficiency. All targets were presented as soluble ectodomains with fusion tags (SmBiT for luciferase, AviTag + His for BLI). Binding was never confirmed against full-length targets or in a cell-surface context.</p><p>For PD-L1, a type I transmembrane protein, cell-surface binding by flow cytometry on PD-L1-expressing cells would confirm the binders work on the native target in its membrane context. Domain truncation can remove steric constraints that block access to the epitope in the full-length protein, and fusion tags near the epitope can interfere with or artificially enhance binding. The IL3 AviTag orientation issue (requiring both C-term and N-term versions) already shows tag placement affects the binding readout.</p><h4><strong>Gap #12: No Run-to-Run Reproducibility Data</strong></h4><p>The paper reports results from single design campaigns per target. The README notes that “in silico success rates vary wildly across targets,” and hallucination-based design is stochastic. Running the same target, same epitope, same parameters a second time could yield a different hit rate.</p><p>Even two independent runs on PD-L1 (their most data-rich target) carried through to BLI would show whether the reported hit rates are reproducible or target-specific outliers.</p><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><h4>Appendix: Germinal’s Three Design Phases</h4><p>Germinal designs CDR sequences through three phases that progressively shift from continuous exploration to discrete sequence selection.</p><p><strong>Logit optimization.</strong> Instead of starting with a fixed amino acid sequence, the model assigns a continuous score (logit) to each of the 20 possible amino acids at every CDR position. These scores are updated iteratively using gradients from both AF-M and IgLM. Because the scores are continuous, the optimization can make smooth adjustments rather than swapping one amino acid for another in discrete steps.</p><p><strong>Softmax annealing.</strong> The logits are converted to probability distributions over amino acids at each position using the softmax function. Early in optimization, the temperature parameter is high, so probabilities are spread across multiple amino acids at each position. As the temperature decreases, the distribution sharpens until one amino acid dominates at each position. This is analogous to simulated annealing in protein structure prediction: high temperature allows broad conformational sampling, low temperature locks in the final state.</p><p><strong>Semi-greedy discrete selection.</strong> In the final phase, the model switches from continuous probabilities to actual amino acid identities. At each iteration, it samples 5 candidate sequences from the current distributions, scores each with the combined AF-M + IgLM objective, and keeps the best. This is “semi-greedy” because it evaluates multiple samples rather than deterministically picking the highest-probability residue at each position, reducing the risk of converging on a local optimum.</p><p>For more info, here are some additional reading materials:<br><strong>• Softmax function and temperature: </strong><a href="https://en.wikipedia.org/wiki/Softmax_function">https://en.wikipedia.org/wiki/Softmax_function</a><br><strong>• Simulated annealing: </strong><a href="https://en.wikipedia.org/wiki/Simulated_annealing">https://en.wikipedia.org/wiki/Simulated_annealing</a><br><strong>• Greedy algorithms:</strong> <a href="https://en.wikipedia.org/wiki/Greedy_algorithm">https://en.wikipedia.org/wiki/Greedy_algorithm</a></p><p><em>Views expressed are my own and do not represent my employer.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b6dfac3140c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Went Through BindCraft’s Affinity Data. Here Are the Gaps I Found.]]></title>
            <link>https://medium.com/@enginyapici/i-went-through-bindcrafts-affinity-data-here-are-the-gaps-i-found-7b90e7880b41?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b90e7880b41</guid>
            <category><![CDATA[protein-design]]></category>
            <category><![CDATA[biotechnology]]></category>
            <category><![CDATA[drug-discovery]]></category>
            <category><![CDATA[drug-design]]></category>
            <category><![CDATA[biotech]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 03 Feb 2026 05:11:47 GMT</pubDate>
            <atom:updated>2026-02-06T15:47:00.930Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://www.epfl.ch/labs/lpdi/">EPFL’s Correia lab</a> published <a href="https://www.nature.com/articles/s41586-025-09429-6">BindCraft in Nature</a> (August 2025), an AlphaFold2-based pipeline for designing protein binders without high-throughput screening. They tested* 212 designs across 12 targets (13 per the supplementary CSV) and found 65 binders (30.7% overall hit rate). Target-specific success rates ranged from 10% (HER2) to 60% (SpCas9).</p><p>The approach: use AF2 multimer to hallucinate binder sequences through backpropagation, optimize non-interface regions with ProteinMPNN, filter predictions with AF2 monomer. The pipeline generates 300–3,000 trajectories per target. The paper recommends filtering to ~100 designs and testing 10 from the top 20, but actual practice varied: most targets had 7–21 designs in the supplementary data with 2–16 tested experimentally. PD-1 was the exception with 53 designs tested*.</p><p>Four binder-target complexes were solved by X-ray crystallography (1.7–3.1 Å RMSD between design and experimental structure). Two additional complexes were solved by cryo-EM. Binding affinities range from sub-nanomolar to low micromolar, though many binders lack affinity measurements.</p><p>They demonstrated functional applications: allergen blocking in patient serum, bacterial toxin neutralization, CRISPR editing inhibition, and AAV retargeting for gene delivery.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PF5QzSSxbLtBVhI3oScd1Q.png" /><figcaption><strong>BindCraft design pipeline and binding results.</strong> (a) Design workflow: AF2 multimer generates binder backbone and sequence, MPNNsol optimizes surface and core while preserving interface, AF2 monomer filters final designs. (b) Twelve protein targets tested. Green regions were used during design, grey regions excluded. Blue boxes: binders found/designs tested. Yellow boxes: measured KD values. Orange boxes: estimated KD** values (poor curve fits). PD-1 binders tested as bivalent Fc fusions. Success rates: 10% (HER2) to 60% (SpCas9)</figcaption></figure><h3>How the model works</h3><p>BindCraft uses AlphaFold2 weights to hallucinate binders through a three-step process:</p><h4><strong>Step 1: AF2 multimer hallucination</strong></h4><p>The pipeline initializes a random sequence for the binder and feeds it through AF2 multimer along with the target structure. Through backpropagation, it optimizes the binder sequence to minimize a composite loss function that includes 9 separate parameters (see the first list in <a href="https://www.nature.com/articles/s41586-025-09429-6#Sec9">the Methods section</a> for the details).</p><p>The optimization runs through four stages: continuous sequence space (75 iterations), softmax probabilities with temperature decay (45 iterations), straight-through estimator (5 iterations), and discrete one-hot sampling (15 iterations). The pipeline randomly swaps between all five AF2 multimer models (model_1_multimer through model_5_multimer, each trained with different random initialization) during hallucination to avoid overfitting to a single model.</p><p>For laypeople: These four stages progressively convert the binder sequence from a flexible representation (where the model can explore many possibilities) to a concrete amino acid sequence. Continuous sequence space allows broad exploration. Softmax probabilities with temperature decay gradually narrows choices. Straight-through estimator begins committing to specific amino acids. Discrete one-hot sampling finalizes the exact sequence. Using all five AF2 models prevents the design from only working with one specific version of the prediction algorithm.</p><p>Target backbone flexibility: Unlike RFdiffusion which fixes the target, BindCraft allows both binder and target backbones to move during design. Target RMSD ranges from 0.5 — 5.5 Å post-design.</p><h4><strong>Step 2: </strong>MPNNsol<strong> sequence optimization</strong></h4><p>The pipeline preserves interface residues (within 4 Å of target) and redesigns the binder core and surface using MPNNsol¹. This generates 20 new sequences per hallucinated trajectory. The goal is to improve expression and solubility while maintaining the designed interface.</p><p>Only two MPNNsol sequences per AF2 trajectory pass to the next stage to promote interface diversity.</p><h4><strong>Step 3: AF2 monomer filtering</strong></h4><p>Each ProteinMPNN-optimized sequence is repredicted using AF2 monomer² (not multimer) in single-sequence mode. The paper doesn’t specify whether they submit the binder sequence alone or concatenate binder + target sequences.</p><p>If testing the binder alone: This checks whether the binder has a stable fold independent of the target, using a model that wasn’t trained on PPIs.</p><p>If testing concatenated sequences: This would be problematic because concatenating two separate proteins would create different folding dynamics that wouldn’t reflect the actual binding interaction.</p><p>The pipeline also predicts the binder alone (without target) to calculate RMSD between bound and unbound states, suggesting the monomer filtering might test binders independently.</p><h4><strong>Step 4: Rosetta scoring and filtering</strong></h4><p>Designs passing AF2 monomer prediction are energy-minimized with Rosetta FastRelax³ (200 iterations) and scored with InterfaceAnalyzer⁴. Nine filters are applied (see the second list in <a href="https://www.nature.com/articles/s41586-025-09429-6#Sec9">the methods section</a> for more details). Designs passing all filters are ranked by i_pTM for experimental testing.</p><h3>Target selection and characteristics</h3><p>The team selected 12 targets (13 including HER2 in the supplementary data) covering diverse protein classes and therapeutic applications:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LFkVayeMl1Uy-n2n415BLA.png" /></figure><p>The diversity covers different fold types (immunoglobulin domains, beta-barrels, alpha-helical bundles), surface properties (highly charged nuclease surfaces, hydrophobic allergen patches), and oligomeric states (monomeric receptors, oligomeric Sas6). BBF-14 tests whether the method works on purely synthetic proteins. It doesn’t have any natural counterparts. The idea is to showcase the model can design based on structure and not memorized patterns.</p><h3>Results</h3><h4><strong>Hit rates and affinities</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nI1NWQmzyaJbKYQ25UCltA.png" /><figcaption>Binding determined by single-concentration SPR (10 µM). Not all binders were characterized for affinity. Most targets have 1 KD measurement; CD45, Der f7, and Sas6 have 2 each; SpCas9 has 6. Values derived from <a href="https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-025-09429-6/MediaObjects/41586_2025_9429_MOESM4_ESM.csv">41586_2025_9429_MOESM4_ESM.csv file</a>.</figcaption></figure><h4><strong>Functional validation</strong></h4><ul><li>BetV1 binder2 blocked ~50% of IgE binding in serum from two out of three birch-allergic patients. The control REGN antibody cocktail⁵ (three antibodies targeting three epitopes) blocked 90%. (<a href="https://www.nature.com/articles/s41586-025-09429-6/figures/3">Figure 3i in the paper</a>)</li><li>CLDN1 binders (binder9 and binder12) protected cells from <em>Clostridium perfringens</em> enterotoxin in a concentration-dependent manner. At the highest concentration tested (++), binder12 reduced cytotoxicity to ~25% (75% cells alive), binder9 to ~45% (55% alive), compared to the CpE inhibitor control at ~40% (60% alive). (<a href="https://www.nature.com/articles/s41586-025-09429-6/figures/2">Figure 2k in the paper)</a></li><li>SpCas9 binders reduced gene editing activity in HEK293T cells. (<a href="https://www.nature.com/articles/s41586-025-09429-6/figures/4">Figure 4d in the paper</a>)</li><li>CbAgo binders slowed DNA cleavage rates 40–80 fold (from kcat 0.004 s⁻¹ to 5–9.8 × 10⁻⁵ s⁻¹). (<a href="https://www.nature.com/articles/s41586-025-09429-6/figures/4">Figure 4f and 4g in the paper</a>)</li><li>AAV retargeting worked. Inserting HER2 and PD-L1 binders into AAV6 capsid (between residues 497 and 498) enabled specific transduction of HER2- and PD-L1-expressing cells. (<a href="https://www.nature.com/articles/s41586-025-09429-6/figures/5">Figure 5 in the paper</a>)</li></ul><h4>Six structures validate atomic-level accuracy</h4><p>The solved structures show excellent agreement with design models:</p><ul><li>BBF-14 + binder4: 1.7 Å backbone RMSD (<a href="https://www.rcsb.org/structure/9HAC">9HAC</a>)</li><li>DerF7 + binder2: 1.7 Å RMSD (two crystal forms at 2.2 Å and 3.0 Å resolution) (<a href="https://www.rcsb.org/structure/9HAE">9HAE</a>)</li><li>DerF21 + binder10: 3.1 Å RMSD (2.6Å resolution) (<a href="https://www.rcsb.org/structure/9HAD">9HAD</a> and <a href="https://www.rcsb.org/structure/9HAF">9HAF</a>)</li><li>SpCas9 + binder3/binder10: Cryo-EM structures with clear density at the REC1 domain (<a href="https://www.emdataresource.org/EMD-53510">EMD-53510</a> and <a href="https://www.emdataresource.org/EMD-53511">EMD-53511</a>)</li></ul><h4><strong>Open-source release with comprehensive documentation</strong></h4><p><a href="https://github.com/martinpacesa/BindCraft">The GitHub repository</a> includes the pipeline, pre-trained models, and detailed protocols. <a href="https://www.nature.com/articles/s41586-025-09429-6#Sec30">The supplementary data</a> contains all 212 design sequences with computational metrics (AF2 scores, interface analysis, etc.) and experimental results.</p><h3>Gaps</h3><h4><strong>1. Affinity data is incomplete and some reported KD values are approximate</strong></h4><p>Many designs show binding without reported affinities. Out of 212 total designs, 20 had reported KD values, and only half of those were without an asterisk. (Please see the Results section above for all KD values). Asterisked values only had estimated KDs due to poor fits.</p><h4><strong>2. Structural validation covers 4 out of 12 targets</strong></h4><p>Any structural coverage is good in computational protein design papers. In this case, authors don’t disclose why they chose those targets or whether crystallization/cryo-EM was attempted for the other eight targets. Selection criteria (binding affinity, crystallizability, structural interest) are not discussed.</p><p>Crystal structures: BBF-14, Der f7, Der f21. Cryo-EM: SpCas9 (two binders).</p><h4><strong>3. </strong>Design selection rationale cannot be verified from published data</h4><p>The paper states they “usually pick 10 designs from the top 20 (ranked by i_pTM) for experimental testing.” in the Methods section. In the supplementary information, they also state this: “We also recommend a final visual inspection of the designs in PyMOL or ChimeraX to exclude designs with high filtering scores but undesirable structural properties, such as wrong binding modes, too small interfaces, or poorly folded protein cores.” This makes sense, but the design selection cannot be independently verified because criteria are not clearly defined. It would have been great to document what exactly they considered to be problematic in the designs they decided to exclude from testing. Here are some suggestions to make the selection more methodical and objective⁶⁻¹²:</p><p><strong>For “wrong binding modes”:<br></strong>• ≥60% of specified hotspot residues contacted (within 4 Å)<br>• Binder center of mass within 10 Å of hotspot centroid<br>• Zero steric clashes (&gt;0.5 Å atomic overlap) when modeled in full biological assembly<br>• For targets with known binders: interface RMSD &lt;5 Å from reference binding mode</p><p><strong>For “too small interfaces”:<br></strong>• Buried surface area ≥600 Å² (standard for protein-protein interfaces)<br>• Interface residues ≥10 per chain<br>• Interface hydrogen bonds ≥5<br>• Shape complementarity ≥0.65<br>• At least one aromatic residue at the interface</p><p><strong>For “poorly folded protein cores”:<br></strong>• Core packing density ≥0.70 (measurable by AREAIMOL or similar)<br>• Buried hydrophobic residues ≥70% of core positions<br>• Buried unsatisfied H-bond donors/acceptors ≤2<br>• Ramachandran favored ≥95%, outliers = 0<br>• Core residues (&gt;90% buried) are hydrophobic (I, L, V, F, Y, W, M)<br>• Sidechain rotamer outliers &lt;5%</p><p><strong>Additional objective criteria they could have used:<br></strong>• Secondary structure content: helices or sheets ≥40% of residues<br>• No long disordered regions: all segments &gt;5 residues have pLDDT &gt;0.70<br>• Electrostatic complementarity at interface (opposite charges within 4 Å)<br>• No large cavities at interface (&gt;50 Å² unfilled volume)</p><p>All of these can be computed automatically from PDB coordinates and would allow reproducibility.</p><h4><strong>4. Several targets show low hit rates or weak affinities</strong></h4><p>HER2: 1/10 (10% success). The single binder worked for AAV retargeting, but this is the worst hit rate in the dataset.</p><p>IFNAR2: 3/20 (15%), best affinity 260 nM. Functional for blocking interferon binding but weaker than other targets.</p><p>Sas6: Micromolar affinities (5.3–5.9 µM). Useful for research probes but not therapeutic development.</p><p>The paper doesn’t analyze why certain targets are harder.</p><h4><strong>5. </strong>Minimal biophysical characterization</h4><p>Regardless of application (therapeutics, research tools, or enabling technologies), protein binders need sufficient stability for expression, purification, storage, and experimental use. The paper provides limited developability data:</p><p><strong>What’s reported:</strong></p><ul><li>Circular dichroism for 3 binders (PD-L1 binder4, IFNAR2 binder5, CD45_b1) showing expected secondary structure at 20°C, 30°C, and 90°C. (<a href="https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-025-09429-6/MediaObjects/41586_2025_9429_MOESM1_ESM.pdf">Supplementary Figure 3</a>)</li><li>SEC-MALS for 4 binders showing oligomeric state: monomeric (IFNAR2 binder5, BBF-14 binder4) or dimeric (PD-L1 binder4, DerF21 binder10, BetV1 binder2).</li></ul><p><strong>What would have been good to see:</strong></p><ul><li><strong>Expression levels</strong>: The paper mentions AF2-hallucinated sequences can have low expression and uses MPNNsol to address this, but doesn’t report actual expression yields for any binder.</li><li><strong>Thermal stability</strong>: CD spectra at multiple temperatures exist for 3 binders, but no melting temperatures (Tm) reported.</li><li><strong>Aggregation propensity</strong>: No data on whether binders aggregate during storage or freeze-thaw cycles.</li><li><strong>Concentration limits</strong>: No data on maximum achievable concentrations before precipitation.</li></ul><p>The dimeric binders (PD-L1 binder4, Der f21 binder10, Bet v1 binder2) engage targets in 1:1 stoichiometry despite solution oligomerization. This could complicate formulation, storage, and interpretation of binding kinetics.</p><p>For practical use in any application, users need to know: Can I express this in reasonable amounts? Will it stay soluble at working concentrations? Can I store it frozen?</p><h4><strong>6. Novelty of the designed interfaces</strong></h4><p>The paper claims the binders are “novel” because they don’t copy binding modes from known protein complexes in the training data. They claim novelty based on TM-score 0.15 versus PPIRef.¹³ But, BetV1 binder2 targets the same epitope as REGN5713 antibody (confirmed by competition, <a href="https://www.nature.com/articles/s41586-025-09429-6/figures/3">Figure 3h</a>) and DerF7 binder2 targets the same site as mouse monoclonal antibodies. This does not invalidate the designs, but it weakens the claim that novelty alone can be inferred from low TM-score against PPIRef.</p><h4><strong>7. Does target flexibility during design help?</strong></h4><p>The paper emphasizes that BindCraft allows target and binder backbone movement during hallucination (unlike RFdiffusion which fixes the target). <a href="https://www.nature.com/articles/s41586-025-09429-6/figures/6">Extended Data Figure 1a</a> shows target RMSD ranges from 0.5–5.5 Å.</p><p>But: how much does this flexibility affect success rates? No ablation study comparing fixed vs. flexible target backbones was performed. The RFdiffusion benchmark shows similar in-silico success rates (<a href="https://www.nature.com/articles/s41586-025-09429-6/figures/6">Extended Data Figure 1c</a>), suggesting target flexibility may not be a critical factor.</p><h4><strong>8. Minor discrepancy in </strong><a href="https://www.nature.com/articles/s41586-025-09429-6/figures/1"><strong>Figure 1b</strong></a></h4><ul><li><strong>PD-L1</strong>: Figure shows “7/9” (7 binders out of 9 tested). CSV shows 11 binders out of 13 tested designs.</li><li><strong>Sas6</strong>: Figure shows “3/9” (3 binders out of 9 tested). CSV shows 3 binders out of 10 tested designs.</li><li><strong>CbAgo</strong>: Figure shows “2/12” (2 binders out of 12 tested). CSV shows 2 binders out of 2 tested designs (10 additional designs in CSV were never tested).</li></ul><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><h3>Footnotes:</h3><p>* Throughout this analysis, “tested” refers to designs with binding data reported in the <a href="https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-025-09429-6/MediaObjects/41586_2025_9429_MOESM4_ESM.csv">41586_2025_9429_MOESM4_ESM.csv file</a> (Binding = 0 or 1). The CSV contains 212 total designs, but 60 have no binding data (Binding = blank).</p><p>** These KD values were reported with an asterisk in the supplementary file <a href="https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-025-09429-6/MediaObjects/41586_2025_9429_MOESM4_ESM.csv">41586_2025_9429_MOESM4_ESM.csv file</a></p><h3>References:</h3><ol><li>Goverde, C. A., Pacesa, M., Goldbach, N., Dornfeld, L. J., Balbi, P. E. M., Georgeon, S., Rosset, S., Kapoor, S., Choudhury, J., Dauparas, J., Schellhaas, C., Kozlov, S., Baker, D., Ovchinnikov, S., Vecchio, A. J., &amp; Correia, B. E. (2024). <em>Computational design of soluble and functional membrane protein analogues</em>. <em>Nature, 631</em>, 449–458. <a href="https://doi.org/10.1038/s41586-024-07601-y">https://doi.org/10.1038/s41586-024-07601-y</a></li><li>Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A. J., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D., Clancy, E., Zielinski, M., Steinegger, M., Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals, O., Senior, A. W., Kavukcuoglu, K., Kohli, P., &amp; Hassabis, D. (2021). <em>Highly accurate protein structure prediction with AlphaFold</em>. <em>Nature, 596</em>, 583–589. <a href="https://doi.org/10.1038/s41586-021-03819-2">https://doi.org/10.1038/s41586-021-03819-2</a></li><li>Leman, J. K., Weitzner, B. D., Lewis, S. M., Adolf‑Bryfogle, J., Alam, N., Alford, R. F., Aprahamian, M., Baker, D., Barlow, K. A., Barth, P., Basanta, B., Bender, B. J., Blacklock, K., Bonet, J., Boyken, S. E., Bradley, P., Bystroff, C., Conway, P., Cooper, S., Correia, B. E., Coventry, B., Das, R., De Jong, R. M., DiMaio, F., Dsilva, L., Dunbrack, R., Ford, A. S., Frenz, B., Fu, D. Y., … Zimmerman, L. (2020). <em>Macromolecular modeling and design in Rosetta: recent methods and frameworks</em>. <em>Nature Methods, 17</em>, 665–680. <a href="https://doi.org/10.1038/s41592-020-0848-2">https://doi.org/10.1038/s41592-020-0848-2</a></li><li>Coventry, B., &amp; Baker, D. (2021). <em>Protein sequence optimization with a pairwise decomposable penalty for buried unsatisfied hydrogen bonds</em>. <em>PLOS Computational Biology, 17</em>(3), e1008061. <a href="https://doi.org/10.1371/journal.pcbi.1008061">https://doi.org/10.1371/journal.pcbi.1008061</a></li><li>Atanasio, A., Franklin, M. C., Kamat, V., Romero Hernandez, A., Badithe, A., Ben, L. H., Jones, J., Bautista, J., Yancopoulos, G. D., Olson, W., Murphy, A. J., Sleeman, M. A., &amp; Orengo, J. M. (2022). <em>Targeting immunodominant Bet v 1 epitopes with monoclonal antibodies prevents the birch allergic response</em>. <em>Journal of Allergy and Clinical Immunology, 149</em>(1), 200–211. <a href="https://doi.org/10.1016/j.jaci.2021.05.038">https://doi.org/10.1016/j.jaci.2021.05.038</a></li><li>Lo Conte, L., Chothia, C., &amp; Janin, J. (1999). <em>The atomic structure of protein-protein recognition sites.</em> <em>Journal of Molecular Biology</em>, 285(5), 2177–2198. <a href="https://doi.org/10.1006/jmbi.1998.2439">https://doi.org/10.1006/jmbi.1998.2439</a></li><li>Janin, J., &amp; Rodier, F. (1995). <em>Protein-protein interaction at crystal contacts. Proteins: Structure, Function, and Bioinformatics</em>, 23(4), 580–587. <a href="https://doi.org/10.1002/prot.340230413">https://doi.org/10.1002/prot.340230413</a></li><li>Lawrence, M. C., &amp; Colman, P. M. (1993). <em>Shape complementarity at protein/protein interfaces. Journal of Molecular Biology</em>, 234(4), 946–950. <a href="https://doi.org/10.1006/jmbi.1993.1648">https://doi.org/10.1006/jmbi.1993.1648</a></li><li>Eriksson, A. E., Baase, W. A., Zhang, X. J., Heinz, D. W., Blaber, M., Baldwin, E. P., &amp; Matthews, B. W. (1992). <em>Response of a protein structure to cavity-creating mutations and its relation to the hydrophobic effect.</em> <em>Science</em>, 255(5041), 178–183. <a href="https://doi.org/10.1126/science.1553543">https://doi.org/10.1126/science.1553543</a></li><li>Fleming, P. J., &amp; Richards, F. M. (2000). <em>Protein packing: dependence on protein size, secondary structure and amino acid composition. Journal of Molecular Biology</em>, 299(2), 487–498. <a href="https://doi.org/10.1006/jmbi.2000.3750">https://doi.org/10.1006/jmbi.2000.3750</a></li><li>Lovell, S. C., Davis, I. W., Arendall, W. B., de Bakker, P. I., Word, J. M., Prisant, M. G., Richardson, J. S., &amp; Richardson, D. C. (2003). <em>Structure validation by Cα geometry: φ,ψ and Cβ deviation. Proteins: Structure, Function, and Bioinformatics</em>, 50(3), 437–450. <a href="https://doi.org/10.1002/prot.10286">https://doi.org/10.1002/prot.10286</a></li><li>Janin, J., Bahadur, R. P., &amp; Chakrabarti, P. (2008). <em>Protein-protein interaction and quaternary structure.</em> <em>Quarterly Reviews of Biophysics</em>, 41(2), 133–180. <a href="https://doi.org/10.1017/S0033583508004708">https://doi.org/10.1017/S0033583508004708</a></li><li><a href="https://github.com/anton-bushuiev/PPIRef">https://github.com/anton-bushuiev/PPIRef</a></li></ol><p><em>Views expressed are my own and do not represent my employer.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b90e7880b41" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Origin-1: Absci’s De Novo Antibody Design Platform]]></title>
            <link>https://medium.com/@enginyapici/origin-1-abscis-de-novo-antibody-design-platform-198d9defb3fa?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/198d9defb3fa</guid>
            <category><![CDATA[protein-design]]></category>
            <category><![CDATA[antibody-design]]></category>
            <category><![CDATA[biotechnology]]></category>
            <category><![CDATA[antibody-engineering]]></category>
            <category><![CDATA[protein-engineering]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 20 Jan 2026 15:29:32 GMT</pubDate>
            <atom:updated>2026-06-02T14:01:49.046Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Disclosure: I previously worked at Absci.</em></p><h4><em>Addendum: Absci’s Updated Origin-1 Preprint</em></h4><p><em>June 2, 2026</em></p><p>Absci posted <a href="https://www.biorxiv.org/content/10.64898/2026.01.14.699389v2">an updated Origin-1 preprint</a> on May 5, 2026. This addendum revisits the five gaps from my original post against the new data. The largest change is in lead optimization: the second round produced 14 sub-nanomolar binders against IL36RA and 3 against COL6A3. The original preprint’s best optimized affinity was 89.4 nM.</p><h4>Gap #1: Micromolar Affinities Have Limited Therapeutic Relevance</h4><p>Status: Closed.</p><p>The second round of lead optimization combined beneficial single mutants into higher-order variants. This produced 14 sub-nanomolar binders against IL36RA and 3 against COL6A3. The top threeIL36RA variants reach KD values of 0.381, 0.503, and 0.762 nM.</p><p>Absci also added additional functional data. In the HEKBlue assay, the highest potency observed was 12.3 nM. A separate assay with CT-26 CCL2 cells confirmed antagonism in mouse cells. The IL36RA binders showed good cross-reactivity against human and mouse IL36RA, with single-digit nanomolar affinity against the mouse ortholog for most designs. One thing that isn’t clear is whether this was an intentional design choice or a result of high similarity between human and mouse IL36RA (93% with UniProt <a href="https://www.uniprot.org/uniprotkb/Q9UBH0/entry#sequences">Q9UBH0</a> and <a href="https://www.uniprot.org/uniprotkb/Q9QYY1/entry#sequences">Q9QYY1</a>). That being said, the sub-nanomolar variants and the cellular antagonism data move IL36RA past the affinity concern I raised in my original post.</p><h4>Gap #2: Developability Data Without Controls</h4><p>Status: Partially addressed.</p><p>Table 4 now reports measured developability values for clinical control antibodies run in the same panel: trastuzumab, cetuximab, briakinumab, bococizumab, and infliximab. Each is assigned a negative or positive control role per assay. Trastuzumab and cetuximab are negative polyreactivity controls; briakinumab and bococizumab are positive. For AC-SINS, trastuzumab is negative and briakinumab and infliximab are positive.</p><p>Table 5 adds developability data for six second-round IL36RA variants. Polyreactivity did not increase substantially despite a &gt;1000-fold affinity gain.</p><p>The control values allow comparison against approved and clinical-stage antibodies. The preprint still does not state numeric pass/fail thresholds or published reference ranges for each assay, so the “therapeutically acceptable” designation rests on the side-by-side control comparison rather than defined cutoffs.</p><h4>Gap #3: Off-Target Antigens Not Identified</h4><p>Status: Closed.</p><p>The off-target proteins are now identified as TL1A and PRLR throughout the figures and methods. Table 1 lists both with PDB IDs, vendors, and catalog numbers. TL1A is trivalent; PRLR is monovalent. Both are unrelated to the ten design targets, so the panel tests specificity against dissimilar proteins rather than structural homologs.</p><h4>Gap #4: Epitope Validation Missing for Key Binders</h4><p>Status: Addressed.</p><p>Absci added a third cryo-EM structure. The IL36RA design complex was solved at 3.1 Å resolution, with a DockQ of 0.87, interface RMSD of 0.78 Å, and CDR RMSDs of 0.642 to 1.006 Å. This confirms the designed epitope on the most therapeutically relevant hit, derived from homology mapping to the IL36γ small molecule binding site (PDB: 6P9E).</p><h4>Gap #5: Text and Figure Inconsistencies</h4><p>Status: Open.</p><p>For the initial hit rate discrepancy, there isn’t any change either in the repo or the paper. Two additional discrepancies appeared: the new sub-nanomolar counts differ from the round-2 optimization files. For IL36RA, the repo lists 13 designs with human KD below 1 nM, while the paper lists 14. This might be due to rounding, as the 14th binder in the CSV file (round2_il36ra_lead_optimization_spr_data.csv) has a KD of 1.01 nM. So I read this as a threshold or rounding difference rather than a data conflict. For COL6A3, the repo lists 2 designs below 1 nM (0.396 and 0.735 nM), and the third-ranked value is 1.015 nM. Figure 14 annotates the third COL6A3 binder at 0.918 nM, which does not appear in round2_col6a3_lead_optimization_spr_data.csv. This difference is the one point I cannot reconcile from the files.</p><p><em>Original analysis published on Jan 20, 2026:</em></p><p>Absci published <a href="https://www.biorxiv.org/content/10.64898/2026.01.14.699389v1">Origin-1</a>, a computational platform for <em>de novo</em> antibody design against epitopes lacking prior structural data. In this post, I summarize what the paper reports and identify gaps in the experimental validation.</p><h3>Platform Overview</h3><p>Origin-1 comprises three components:</p><ul><li><strong>AbsciDiff</strong>: Diffusion-based structure generation, fine-tuned from Boltz-1. The model generates antibody-antigen complexes conditioned on epitope specification. Key modifications include epitope conditioning through feature masking, a sequence hypothesis module, and template support. Training used 10,045 antibody-antigen structures from PDB with temporal and sequence homology filters.</li><li><strong>IgDesign2</strong>: CDR sequence design model with a GNN encoder, causal transformer decoder, and IgBert³ refinement. Takes structural predictions from AbsciDiff and generates CDR sequences compatible with the predicted binding geometry.</li><li><strong>AbsciBind</strong>: Scoring protocol combining ipTM and a custom Antibody-Aligned ipTM metric. Both metrics were developed by Absci for this paper. The score is derived from AF_Unmasked¹ with modifications: amino acid tokens retained, side chains masked except Cβ, and inter-chain template distances enabled. Designs with AbsciBind Score ≥0.5 were considered high-confidence.</li></ul><p><strong>Three-stage filtering pipeline:</strong></p><p><strong>Wide Structure Search</strong>: 3,360 design specifications sampled (combinations of frameworks, CDR lengths, epitope residues). AbsciDiff generated 80,640 structures (24 diffusion samples per specification), selected 10,080 for scoring. IgDesign2 generated one sequence per structure. Top 10% by AbsciBind Antibody-Aligned ipTM Score advanced (1,008 design specifications, oversampled 10× to 10,080 specifications for Deep Search).</p><p><strong>Deep Structure Search</strong>: 10,080 design specifications from Wide Search. AbsciDiff generated 241,920 structures, selected 30,240 for scoring. Pooled with Wide Search structures (40,320 total). Filtered by L-RMSD &lt;5Å between designed and predicted structures. Top 500 structures by Antibody-Aligned ipTM Score and Intersection Score advanced to Sequence Search. Structural diversity enforced through agglomerative clustering (maximum 30% from single cluster).</p><p><strong>Sequence Search</strong>: 320 sequences generated per structure (500 structures = 160,000 sequences). Filtered for sequence liabilities and deduplicated. Top 20 sequences per structure scored with AbsciBind. Final selection: 95 designs per target, balancing AbsciBind Score with diversity constraints (maximum 3 replicates per HCDR3 sequence, maximum 30% from single structural cluster).</p><p><strong>Per target totals</strong>: ~322,000 structures generated, ~40,000 structures scored, ~200,000 sequences generated, ~50,000 sequences scored with AbsciBind.</p><h3>Targets, Hit Rates, and Hit Affinities</h3><p>Ten human protein targets were selected based on: high structural resolution (&lt;3.5 Å), few missing residues, commercial antigen availability, no existing antibody-antigen complex structures in PDB, and ≤60% sequence identity to any protein with a known complex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aN0sLOpzp3tCHxeVwjb_0A.png" /><figcaption>* Paper text reports 9 total SPR hits (COL6A3: 3, AZGP1: 4, CHI3L2: 1, IL36RA: 1), but GitHub repository contains binding data for only 5 designs.</figcaption></figure><h4>Hit affinities:</h4><p>The paper states they got “9 total SPR hits” but only reports affinities for 5 binders. The <a href="https://github.com/AbSciBio/origin-1">GitHub repository</a> confirms only 5 designs as binders (marked “True” in ‘<a href="https://github.com/AbSciBio/origin-1/blob/main/de_novo_spr_data.csv">de_novo_spr_data.csv</a>’ and have SPR KD values reported in ‘<a href="https://github.com/AbSciBio/origin-1/blob/main/lead_optimization_spr_data.csv">lead_optimization_spr_data.csv</a>’):</p><p><strong>COL6A3: </strong>1 hit, 1.94 µM.</p><p><strong>AZGP1: </strong>2 hits, 1.42 and 4.50 µM. No BLI confirmation data for the second hit.</p><p><strong>CHI3L2: </strong>1 hit, 3.46 µM. Failed BLI confirmation.</p><p><strong>IL36RA: </strong>1 hit, 6.07 µM. Failed BLI confirmation.</p><h3>Structural Validation</h3><p>Cryo-EM structures were solved for COL6A3 and AZGP1 complexes:</p><h4><strong>COL6A3 (3.0 Å resolution):</strong></h4><ul><li>All-atom global RMSD: 2.56 Å</li><li>Interface RMSD: 0.96 Å</li><li>DockQ²: 0.83</li><li>CDR RMSDs: 0.661–1.486 Å</li></ul><h4><strong>AZGP1 (3.1 Å resolution):</strong></h4><ul><li>All-atom global RMSD: 1.79 Å</li><li>Interface RMSD: 1.35 Å</li><li>DockQ: 0.73</li><li>CDR RMSDs: 0.751–2.056 Å</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mGNbRCTx-EHudB2-15NWMQ.jpeg" /><figcaption><strong>COL6A3 binding validation and structural confirmation for the strongest de novo hit (1.9 µM parent affinity).</strong> (A) SPR sensorgrams show concentration-dependent binding to COL6A3 in mAb format with no cross-reactivity to off-targets. Note the biphasic kinetics. (B) BLI confirms binding in Fab format when antibody is immobilized; binding failed when antigen was immobilized (data not shown). (C) SEC shows complex formation between COL6A3 and Fab, though complex formation required Nanodaptor and Kappabulk stabilizers. (D) Cryo-EM structure at 3.0 Å resolution shows 0.96 Å interface RMSD between computational model (red/black/gray) and experimental structure (salmon), validating the predicted binding mode. (E) Individual CDR overlays demonstrate sub-2Å structural accuracy across all CDRs.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M6Pzow-1xqDzEYF7UQSAQw.jpeg" /><figcaption><strong>AZGP1 binding validation and structural confirmation for the strongest de novo hit (1.4 µM parent affinity).</strong> (A) SPR sensorgrams show concentration-dependent binding to AZGP1 in mAb format with no cross-reactivity to off-targets. (B) BLI confirms binding in Fab format when antibody is immobilized. Unlike COL6A3, AZGP1 designs also bound in the flipped orientation with antigen immobilized (data not shown in figure). (C) SEC shows complex formation between AZGP1 and Fab without requiring additional stabilizers. (D) Cryo-EM structure at 3.1 Å resolution shows 1.35 Å interface RMSD between computational model (red/black/gray) and experimental structure (salmon), validating the predicted binding mode. DockQ score of 0.73 indicates good model quality. (E) Individual CDR overlays demonstrate structural prediction accuracy across all six CDRs. Despite the structural validation, this design was flagged for multiple developability concerns: self-association (25.7 nM AC-SINS), polyreactivity against both insulin (15.8 RNS) and DNA (8.5 RNS), and hydrophobicity. The 1.4 µM affinity is the strongest in the paper but remains well below what would typically justify downstream therapeutic development.</figcaption></figure><p>The COL6A3 complex required Nanodaptor and Kappabulk stabilization for cryo-EM grid preparation. In BLI, COL6A3 showed binding only in the antibody-immobilized orientation; it failed to bind when antigen was immobilized.</p><p>CHI3L2 and IL36RA binders (including optimized variants with improved affinity and functional activity) do not have experimental structures.</p><h3>Lead Optimization</h3><p>Single-mutant libraries (94 variants + 1 parent) were designed using AbsciBind scores combined with ESM ensemble and AbLang2. The <a href="https://github.com/AbSciBio/origin-1">GitHub repository</a> contains KD values for 354 optimization variants.</p><p>Optimization hit rates (variants showing actual improvement over parental molecules):</p><ul><li>AZGP1-A: 21/94 (22%)</li><li>AZGP1-B: 28/94 (30%)</li><li>IL36RA: 31/94 (33%)</li><li>CHI3L2: 28/94 (30%)</li><li>COL6A3: 24/94 (26%)</li></ul><p>Of the 94 variants tested per parent, 22–33% showed improved affinity over the parental molecule. But of the 354 optimization variants with KD values, 324 remained above 1 µM. Nine variants reached sub-500 nM affinity, and just three broke 200 nM.</p><h3>Computational Benchmark</h3><p>AbsciGen was compared to a modified version of RFantibody. Modifications were required because RFantibody’s default coordinate initialization creates chain breaks for full-length antibodies, and public examples contained actual PDB loop coordinates.</p><p>Results using AbsciBind Score ≥0.5 threshold:</p><ul><li>AbsciGen: 28.37% of designs passed</li><li>RFantibody: 1.49% of designs passed</li></ul><p>AbsciBind Score is the average of ipTM and Antibody-Aligned ipTM, both metrics developed by Absci for this paper.</p><h3>Methodological Transparency</h3><p>The level of detail in this paper exceeds typical industry publications. The authors describe:</p><h4><strong>Model architecture and training:</strong></h4><ul><li>AbsciDiff’s modifications to Boltz-1 (feature masking, epitope conditioning, sequence hypothesis module, template support)</li><li>IgDesign2’s architecture (GNN encoder, causal transformer decoder, IgBert refinement)</li><li>Training data curation (AbData pipeline, 10,045 PDB structures, inclusion/exclusion criteria, temporal and sequence homology filters)</li><li>Cropping strategy (512 residue maximum, Fv always included, constant domains excluded)</li><li>Fine-tuning parameters (20 learning rate steps, batch size 64, 10 epochs)</li></ul><h4><strong>Design generation pipeline:</strong></h4><ul><li>Three-stage filtering (Wide Structure Search =&gt; Deep Structure Search =&gt; Sequence Search)</li><li>Design specification parameters (HCDR3 length 8–26, LCDR3 length 8–10, epitope subsampling 45–90%)</li><li>3360 design specifications sampled per target, yielding ~40,000 structures and ~170,000 sequences</li><li>Framework selection rationale (trastuzumab, relatlimab, sotrovimab, and dupilumab to cover diverse germlines)</li></ul><h4><strong>Epitope selection:</strong></h4><ul><li>Criteria for each target (surface geometry, curvature, hydrophobic patches, structured regions)</li><li>Specific residues selected per epitope (Figure 6 in the paper)</li><li>IL36RA epitope derived from structural homology to IL36γ small molecule binding site (PDB: 6P9E)</li><li>Acknowledgment of AZGP1 PDB entry missed during selection</li></ul><h4><strong>Scoring protocol:</strong></h4><ul><li>AbsciBind derivation from AF_Unmasked</li><li>Modifications to AFM protocol (amino acid tokens retained, side chains masked except Cβ, inter-chain template distances enabled)</li><li>Benchmark against six reference approaches with performance comparison</li></ul><h4><strong>Experimental methods:</strong></h4><ul><li>SPR parameters (chip type, buffer composition, regeneration conditions, analysis software, exclusion criteria)</li><li>BLI protocol (sensor types, baseline, association/dissociation times)</li><li>Developability assays (methods for each assay, system suitability controls)</li><li>Cryo-EM data collection and processing (microscope settings, software pipeline)</li></ul><h3>Fair Benchmark Against RFantibody</h3><p>The authors identified problems with RFantibody that would have artificially inflated their comparative results:</p><ol><li><strong>Chain breaks:</strong> Running RFantibody without modification produced chain breaks in predicted loop structures and glycine-dominated sequences from ProteinMPNN.</li><li><strong>Information leakage:</strong> The public RFantibody release includes actual loop coordinates from PDB entries in the provided framework files. Using these coordinates in a “de novo” benchmark would leak structural information.</li><li><strong>Initialization bias:</strong> RFantibody’s default coordinate initialization causes spatial biases to persist through the diffusion process.</li></ol><p>Rather than using RFantibody out of the box (which would have made Origin-1 look better by comparison), the authors:</p><ul><li>Preprocessed structures to place CDR loop coordinates at the origin</li><li>Applied idealized initialization based on loop dilation logic</li><li>Documented the modifications in Supplement §7.3</li><li>Showed results comparing original vs. modified RFantibody (Supplementary Figure 6)</li></ul><p>The paper explicitly states this was done “to ensure a fair benchmark.” They could have published a comparison against the unmodified RFantibody showing larger performance gaps. Choosing to fix the competitor’s implementation before comparison is commendable.</p><p>The methods sections are detailed and the RFantibody benchmark was handled fairly: they fixed the coordinate initialization problem before comparison rather than capitalizing on a broken baseline. But detailed protocols for generating micromolar binders don’t make those binders therapeutically useful. Several gaps and inconsistencies make the interpretation of the results difficult.</p><h3>Gap #1: Micromolar Affinities Have Limited Therapeutic Relevance</h3><p>Parent design affinities from SPR:</p><ul><li>COL6A3: 1.94 µM</li><li>AZGP1: 1.42 and 4.50 µM</li><li>CHI3L2: 3.46 µM</li><li>IL36RA: 6.07 µM</li></ul><p>Best affinities after optimization:</p><ul><li>IL36RA: 89.4 nM</li><li>COL6A3: 174 nM</li><li>AZGP1: 193 nM</li></ul><p>CHI3L2 and IL36RA hits showed SPR signals but could not be confirmed by BLI prior to optimization. The authors attributed this to low affinity (KD &gt; 2 µM). However, the second AZGP1 hit (4.50 µM) is weaker than CHI3L2 (3.46 µM) yet was reportedly confirmed by BLI (though no data for this second hit appears in Figure 10).</p><p>For context, here’s what canonical antibody discovery methods typically produce:</p><h4><strong>Phage display</strong></h4><p>Initial hits from naive libraries typically range from 10–500 nM. After 3–5 rounds of affinity maturation, leads routinely reach single-digit nanomolar or picomolar affinities. A standard phage campaign costs $10,000–50,000 and takes 3–6 months.</p><h4><strong>Hybridoma</strong></h4><p>Immunization-based campaigns produce polyclonal responses where lead candidates typically show 1–100 nM affinity after initial screening. Further optimization can reach sub-nanomolar.</p><h4><strong>Yeast display</strong></h4><p>Similar to phage, with initial hits in the 10–100 nM range and affinity-matured leads reaching picomolar affinities through iterative sorting.</p><p><strong>Approved therapeutic antibodies:</strong></p><ul><li>Trastuzumab (Herceptin) binds HER2 at ~5 nM</li><li>Adalimumab (Humira) binds TNF-α at ~100 pM</li><li>Pembrolizumab (Keytruda) binds PD-1 at ~29 pM.</li></ul><p><strong>Other <em>de novo</em> AI platforms:</strong></p><ul><li><a href="https://medium.com/@enginyapici/i-tried-to-poke-holes-in-chai-2s-antibody-design-paper-here-s-what-i-found-7e51f5581c7d">Chai-2</a> achieves sub-nanomolar affinities (453 pM for CCR8) with 28% hit rates</li><li><a href="https://medium.com/@enginyapici/jam-2-performance-metrics-and-validation-gaps-51f40a6511f2">JAM-2</a> reports 39% hit rates (VHH-Fc format) with initial affinities in the low-nanomolar range</li><li><a href="https://medium.com/@enginyapici/low-immunogenicity-in-human-panels-examining-latent-x2s-evidence-9c540b8f49ba">Latent-X2</a> reports 21% hit rates for VHH designs with affinities in the 3–50 nM range</li></ul><p>Origin-1’s parent designs (1.42–6.07 µM) are 3–600x weaker than typical phage display hits before any optimization. The optimized variants (89–193 nM) are comparable to unoptimized display library outputs, not optimized leads.</p><p>In most therapeutic antibody programs, molecules do not advance to functional characterization, <em>in vivo</em> studies, or lead optimization until they reach single-digit nanomolar affinity. A 100 nM binder would typically be considered a starting point for optimization, not a validated hit. The gap between Origin-1’s best output (89 nM) and therapeutic relevance (sub-nanomolar) represents at least one additional optimization campaign.</p><h3>Gap #2: Developability Data without Controls</h3><p>The paper states: “COL6A3, CHI3L2, and IL36RA binders met therapeutically acceptable criteria for these developability properties, aside from one hydrophobicity flag associated with the IL36RA binder.”</p><p>The methods section states: “All the assays were evaluated for system suitability checks in each run or plate as determined by the performance of a negative control, trastuzumab for all assays, and at least one positive control such as Infliximab for AC-SINS, Bococizumab and Briakinumab for polyreactivity, or BSA and Insulin for HIC. For DLS and nanoDSF, standard particles solution and lysozyme standard were used for system suitability checks in each run, respectively.”</p><p>System suitability controls verify that assays are functioning correctly. They do not provide reference values for interpreting results.</p><p>The paper does not report:</p><ul><li>Any clinical antibody values run in parallel</li><li>Published reference ranges for “therapeutically acceptable” criteria</li><li>Thresholds used to define pass/fail for each assay</li></ul><p>Without clinical antibody benchmarks in the same experiments, the claim that designs “met therapeutically acceptable criteria” cannot be independently evaluated. A polyreactivity RNS of 2.5 or AC-SINS value of 4.9 nm may or may not be acceptable depending on the reference range for approved antibodies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_8gjkXgFVS4mYYycRz2lJQ.png" /><figcaption><strong>Developability results for confirmed binders.</strong> AZGP1-A showed flags for self-association, hydrophobicity, and polyreactivity. IL36RA showed one hydrophobicity flag. COL6A3 and CHI3L2 passed all assays. Control antibody values (trastuzumab, infliximab, bococizumab, briakinumab) were measured but not reported, so absolute interpretation of these values is not possible. <strong>HIC</strong> = Hydrophobic Interaction Chromatography; <strong>RRT</strong> = Relative Retention Time; <strong>AC-SINS</strong> = Affinity-Capture Self-Interaction Nanoparticle Spectroscopy; <strong>DSF</strong> = Differential Scanning Fluorimetry; <strong>DLS</strong> = Dynamic Light Scattering; <strong>PDI</strong> = Polydispersity Index; <strong>N.D.</strong> = Not Detected.</figcaption></figure><h3>Gap #3: Off-Target Antigens Not Identified</h3><p>The paper states designs were screened “against at least two non-antigen commercial protein targets (‘off-targets’) to assess polyspecificity.” Figures 9–12 label these as “Off-Target 1” and “Off-Target 2” without identifying the proteins.</p><p>Knowing the off-target panel composition is relevant for interpreting specificity claims. Two proteins selected to be structurally dissimilar from the target provide weaker evidence of specificity than proteins with similar folds or overlapping epitope features. Standard practice in antibody characterization is to report the identity of all proteins tested.</p><h3>Gap #4: Epitope Validation Missing for Key Binders</h3><p>The central claim of Origin-1 is designing antibodies against user-specified novel epitopes. Validating that designs actually engage the intended epitope (not just bind somewhere on the target) is essential to this claim.</p><p>Cryo-EM structures for COL6A3 and AZGP1 confirm the designed epitopes were hit. But COL6A3 and AZGP1 are also the weakest therapeutic candidates, where COL6A3 required scaffold stabilization (Nanodaptor and Kappabulk) just to obtain a structure, and AZGP1 had the most developability flags.</p><p>More importantly, no cryo-EM structures were solved for IL36RA or CHI3L2. IL36RA is the most therapeutically relevant hit: best optimized affinity (89.4 nM), demonstrated functional antagonism in a cell-based assay, and fewest developability flags. The IL36RA epitope was derived from homology mapping to a small molecule binding site on IL36γ (PDB: 6P9E). Without a structure, we don’t know if IL36RA binders engage this intended site or achieve antagonism through a different binding mode.</p><p>No epitope binning or competition assays were performed for any target.</p><h3>Gap #5: Text and Figure Inconsistencies</h3><h4><strong>Hit count discrepancy</strong></h4><p>The paper states “9 total SPR hits” across all targets (COL6A3: 3, AZGP1: 4, CHI3L2: 1, IL36RA: 1). BLI confirmed only 3 of these (COL6A3: 1, AZGP1: 2), but Figure 10 shows BLI data for only one AZGP1 hit. CHI3L2 and IL36RA showed “limited mAb BLI binding” for parent designs and required optimization before confirmation. The GitHub repository ‘<a href="https://github.com/AbSciBio/origin-1/blob/main/de_novo_spr_data.csv">de_novo_spr_data.csv</a>’ file contains 5 binders (COL6A3: 1, AZGP1: 2, CHI3L2: 1, IL36RA: 1).</p><h4>CHI3L2 complexation claim contradicts figure</h4><p>Figure 11C caption states “Complexation experiment confirms that optimized variant 1, in Fab format, binds to CHI3L2 in solution.” The SEC chromatogram in Figure 11C shows no complex peak. The antigen-alone peak isn’t visible (possibly omitted from the figure). The main peak labeled as complex has nearly the same retention time as Fab alone. The text claims complexation; the figure doesn’t support it.</p><h4>COL6A3 binding complications</h4><p>SPR sensorgrams show biphasic binding kinetics. The methods state data was cropped to 375 seconds for analysis. Biphasic kinetics often indicate multiple binding models, avidity effects, or heterogeneous samples. BLI confirmation only worked in antibody-immobilized orientation; it failed when antigen was immobilized. The authors attribute this to “small size of the antigen and potential tag interference impacting the binding interface when the antigen was immobilized on the probe.”</p><h4><strong>AZGP1-B BLI data missing</strong></h4><p>The paper states “BLI confirmed one hit against COL6A3 (Figure 9), and two hits against AZGP1 (Figure 10).” in section 3.3. Figure 10 caption describes “the top design” (singular), and panels show BLI data for one AZGP1 binder only. No BLI data for the second AZGP1 hit (AZGP1-B, 4.50 µM per GitHub) appears in the figure. The paper also states CHI3L2 and IL36RA could not be confirmed by BLI due to “low affinity (KD &gt; 2 µM).” CHI3L2 parent affinity is 3.46 µM, which is tighter than AZGP1-B at 4.50 µM. A 4.50 µM binder passing BLI while a 3.46 µM binder fails is inconsistent with the stated sensitivity limit.</p><h3>Summary</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d69NEGzupeaWj-zETNz7HA.png" /></figure><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><h3>References:</h3><ol><li>Mirabello, C., Wallner, B., Nystedt, B., Azinas, S., &amp; Carroni, M. (2024). <em>Unmasking AlphaFold to integrate experiments and predictions in multimeric complexes</em>. <em>Nature Communications, 15</em>, 8724. <a href="https://doi.org/10.1038/s41467-024-52951-w">https://doi.org/10.1038/s41467-024-52951-w</a></li><li>Basu, S., &amp; Wallner, B. (2016). <em>DockQ: A quality measure for protein-protein docking models</em>. <em>PLOS ONE, 11</em>(8), e0161879. <a href="https://doi.org/10.1371/journal.pone.0161879">https://doi.org/10.1371/journal.pone.0161879</a></li><li>Kenlay, H., Dreyer, F. A., Kovaltsuk, A., Miketa, D., Pires, D., &amp; Deane, C. M. (2024). <em>Large scale paired antibody language models. PLOS Computational Biology, 20(12)</em>, e1012646. <a href="https://doi.org/10.1371/journal.pcbi.1012646">https://doi.org/10.1371/journal.pcbi.1012646</a></li></ol><p><em>Views expressed are my own and do not represent my employer.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=198d9defb3fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RFdiffusion3: Atom-Level Protein Design at Scale]]></title>
            <link>https://medium.com/@enginyapici/rfdiffusion3-atom-level-protein-design-at-scale-03c2208785d4?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/03c2208785d4</guid>
            <category><![CDATA[therapeutics]]></category>
            <category><![CDATA[computational-biology]]></category>
            <category><![CDATA[ai-in-healthcare]]></category>
            <category><![CDATA[protein-design]]></category>
            <category><![CDATA[synthetic-biology]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Wed, 14 Jan 2026 14:54:19 GMT</pubDate>
            <atom:updated>2026-01-14T18:16:38.163Z</atom:updated>
            <content:encoded><![CDATA[<p>Baker Lab released <a href="https://www.ipd.uw.edu/2025/12/rfdiffusion3-now-available/">RFdiffusion3</a>, an all-atom diffusion model for designing proteins that interact with DNA, ligands, and other biomolecules. The paper reports improved performance on multiple benchmarks, 10x faster inference than RFdiffusion2, and experimental validation with DNA binders and enzymes.</p><p>RFdiffusion3 treats every atom explicitly (backbone and sidechains) using a 14-atom-per-residue representation. This removes the hybrid residue-atom approach from RFdiffusion2 and enables direct conditioning on hydrogen bond donors/acceptors, burial states, and other atomic constraints.</p><p>The DNA binder screen tested 5 designs and found 1 with EC50 = 5.89 ± 2.15 µM. The enzyme screen tested 190 cysteine hydrolase designs and found 35 multi-turnover catalysts, with the best achieving kcat/Km = 3557 M⁻¹s⁻¹.</p><p>The experimental validation falls short of standards this group established in earlier work. RFdiffusion1 included negative controls (H52A mutants that abolished metal binding in 17/17 cases), experimental structures (2.9 Å cryo-EM of HA binder with 0.63 Å RMSD to design model), rigorous binding measurements (KD values via BLI titration), and specificity testing (competition assays for IL-7Rα binders).¹ RFdiffusion2 metallohydrolases included negative controls for the most active design (H→A mutants for ZETA_1), uncatalyzed rate baselines for comparison, and structural validation.³</p><p>This paper reports enzyme kinetics with error bars and binding curves to determine EC50 values, which is good. But it’s missing key controls: no catalytically-dead mutants for the enzymes, no binding-dead mutants for the DNA binder, no experimental structures (cryo-EM or crystallography) to confirm the designed interfaces, and no specificity controls showing the DNA binder recognizes its target sequence rather than DNA generally.</p><h3>How RFdiffusion3 Works</h3><p>RFdiffusion3 differs from earlier versions in how it represents protein structure:</p><p><strong>RFdiffusion1</strong> operated at the amino acid level. Each residue represented by backbone atoms (N, Cα, C, O). This worked for designing protein monomers, assemblies, and protein-protein interfaces, but couldn’t specify interactions with individual sidechain atoms.</p><p><strong>RFdiffusion2</strong> remained residue-level but allowed conditioning on a small number of critical sidechain atoms (for enzyme active sites or ligand binding). You could tell it “put a histidine here with this sidechain orientation,” but the diffusion process still operated on residues, not individual atoms. This limited the model’s ability to create additional sidechain-ligand contacts beyond what you explicitly specified.</p><p><strong>RFdiffusion3</strong> diffuses all atoms explicitly, using a 14-atom-per-residue representation that includes up to 10 sidechain heavy atoms. This removes the hybrid residue-atom approach from RFdiffusion2, where the diffusion process still operated on residues rather than individual coordinates. By treating every atom as a diffusing particle, the model can generate protein-ligand interactions de novo rather than just positioning pre-specified atoms.</p><p>This enables <strong>atom-level hotspot specification</strong>: you can now specify which atoms (not just which residues) should be contacted. Earlier methods let you say “design a binder to this surface patch”. RFdiffusion3 lets you say “make contacts to the carbonyl oxygen of residue 47 and the sidechain nitrogen of residue 52.” This precision is critical for inhibiting specific protein-protein interactions or blocking enzyme active sites where single-atom contacts determine binding specificity.</p><p>Training data includes PDB structures through December 2024 plus AlphaFold2 distillation structures predicted in Hsu et al.²²</p><p>Key conditioning capabilities:</p><ul><li><strong>Hydrogen bonds</strong>: Specify which atoms donate/accept H-bonds</li><li><strong>Burial</strong>: Control solvent accessibility per atom</li><li><strong>Center-of-mass</strong>: Position functional sites relative to protein core</li><li><strong>DNA co-diffusion</strong>: Generate DNA shape alongside protein</li></ul><p>The model is open source (MIT license) and available at <a href="https://github.com/RosettaCommons/RFdiffusion">github.com/RosettaCommons/RFdiffusion</a>. This enables independent validation and extension by other groups.</p><h3>Design Workflow</h3><p>RFdiffusion3 generates backbone structures, not sequences. The full design pipeline requires three steps:</p><ol><li><strong>RFdiffusion3 generates backbone geometry</strong><br>The model outputs 3D coordinates for all atoms (backbone + sidechains). But the amino acid sequence inferred from these coordinates isn’t optimized. The model “only sees the final backbone structure in the last diffusion step.”</li><li><strong>ProteinMPNN designs sequences</strong><br>ProteinMPNN (for protein-only designs) or LigandMPNN (for designs with DNA, ligands, or metals) takes the backbone and generates amino acid sequences predicted to fold into that structure. The paper generates 4–8 sequences per backbone depending on the task.</li><li><strong>AlphaFold3 validates designs</strong><br>AF3 predicts the structure from each sequence. Success criteria vary by task (typically 1.5–2.5Å RMSD to the designed backbone, plus additional metrics like PAE and pTM scores).</li></ol><h3>Results Summary</h3><p>The paper tests RFdiffusion3 across four design categories:</p><h4><strong>Protein-binding proteins</strong></h4><p>In silico benchmark comparing RFdiffusion3 to RFdiffusion1 across 5 targets using AlphaProteo filters. RFdiffusion3 shows higher success rates and generates more diverse binding modes. No experimental validation.</p><h4><strong>Protein-nucleic acid interactions</strong></h4><p>DNA binder design tested experimentally. Generated 400 designs per target DNA sequence, tested 5 designs. Found 1 binder (DBRFD3) with EC50 = 5.89 ± 2.15 µM. DNA co-diffusion tested computationally. Model generates DNA shape alongside protein, with 7–9% of designs showing AF3-predicted DNA binding.</p><h4><strong>Small molecule binders</strong></h4><p>Tested on 4 ligands (FAD, SAM, IAI, OQO). Success rates: ~10% (FAD), ~30% (SAM), ~33% (IAI), ~42% (OQO). Comparison shows improvement over RFdiffusionAA (1–10% success rates). No experimental validation. All results are in silico comparisons with AF3 predictions. Interestingly, success rates don’t correlate with training set frequency: FAD (<a href="https://www.rcsb.org/ligand/FAD">2,852 PDB structures</a>) shows the lowest success (~10%), while OQO (<a href="https://www.rcsb.org/ligand/OQO">1 structure</a>) shows among the highest (~42%). This suggests performance depends more on ligand properties and binding mode complexity than training set representation, though more ligands would be needed to confirm this pattern.</p><h4><strong>Enzyme design</strong></h4><p>Cysteine hydrolase active site scaffolding. Tested 190 designs experimentally, found 35 with multi-turnover activity (18% hit rate). Best design (C6) shows kcat/Km = 3557 ± 624 M⁻¹s⁻¹. Paper claims this exceeds prior computational designs for the same reaction.</p><p>The experimental work focuses on DNA binding (1/5 designs tested) and enzymes (35/190 active). Protein-protein binders and small molecule results are purely computational.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AnTDfMXV0gQIf-wAQ_w1qQ.png" /><figcaption><strong>RFdiffusion3 in silico benchmarking across four design challenges.</strong> (a) Protein-protein binders against 5 therapeutic targets. RFD3 generates more successful designs than RFdiffusion1 (solid bars = unique clusters at TM-score 0.6; hatched = total successful backbones; 400 tested per target). Success defined by AlphaFold3 metrics: minimum PAE &lt; 1.5, binder pTM &gt; 0.8, RMSD &lt; 2.5Å. (b) DNA-binding proteins for 3 targets. Color-coded by DNA-aligned RMSD thresholds when predicted with AF3. Testing both fixed DNA conformations (+) and de novo DNA shape prediction (-). © Small molecule binders for FAD, SAM, IAI, OQO compared to RFdiffusionAA baseline. Success rates shown as fraction of 400 backbones passing AF3 filters (ligand RMSD &lt; 5Å, backbone RMSD &lt; 1.5Å, min PAE &lt; 1.5, iPTM &gt; 0.8). (d) Enzyme active site scaffolding on AME benchmark with increasing complexity (1–4 residue islands). Validated with Chai-1 predictions requiring motif all-atom RMSD &lt; 1.5Å. All panels show design models (teal/colors) with structure predictions overlaid (gray). Note: These are computational predictions, not experimental validation.</figcaption></figure><h3>Issue #1: DNA Binder Validation Has Critical Gaps</h3><p>The paper reports 1 successful DNA binder from 5 tested designs (20% success rate). This binder, DBRFD3, binds target sequence CGAGAACATAGTCG with EC50 = 5.89 ± 2.15 µM.</p><h4><strong>Missing specificity controls</strong></h4><p>The paper designed DBRFD3 to bind sequence CGAGAACATAGTCG and shows it binds with EC50 = 5.89 µM. But there are no controls to demonstrate this binding is sequence-specific rather than promiscuous.</p><p>To show specificity, the paper needs two types of controls:</p><ol><li><strong>Test DBRFD3 against unrelated DNA sequences</strong><br>Does DBRFD3 bind other sequences with similar affinity? If it binds ATATAT or GCGCGC or random sequences at 5–10 µM, then the binding isn’t specific to CGAGAACATAGTCG. It’s just a sticky protein that binds DNA.</li><li><strong>Test other designs against CGAGAACATAGTCG</strong><br>Design proteins against different target sequences (e.g., ATATAT or GCGCGC), then test those against CGAGAACATAGTCG. If proteins designed for other targets also bind CGAGAACATAGTCG, that shows the designs aren’t sequence-specific.</li></ol><p>Neither control appears in the paper. Without them, we can’t tell whether RFdiffusion3 designed a sequence-specific DNA binder or just generated a protein with nonspecific DNA affinity.</p><p>The 14bp target sequence has:</p><ul><li>50% GC content (genomic average)</li><li>GAGA motif that is a common transcription factor binding site</li></ul><p>The supplemental shows DBRFD3 makes major groove contacts to adenine (N36) and guanine (R39) residues, but doesn’t test whether mutating these bases affects binding. Standard validation would include:</p><ul><li>Binding to scrambled sequences with matched base composition</li><li>Single base-pair substitutions at predicted contact positions</li><li>Mutagenesis of contact residues (does N36A or R39A abolish binding?)</li></ul><p>These experiments distinguish sequence recognition from electrostatic attraction to DNA backbone.</p><h4><strong>No structural validation</strong></h4><p>The paper presents an AF3 model of DBRFD3 bound to DNA with 1.83Å RMSD. The supplemental claims DBRFD3 makes major groove contacts to adenine (N36) and guanine (R39) residues. But there’s no experimental structure confirming these contacts.</p><p>RFdiffusion1 obtained 2.9 Å cryo-EM structures of designed binders in complex with targets, confirming atomic-level accuracy (0.63 Å RMSD between experimental structure and design model for the HA binder). The absence of experimental structures here is a regression from their own earlier standards.</p><h4><strong>Affinity context missing</strong></h4><p>The 5.89 µM KD is 20–100x weaker than typical transcription factor affinities (1–100 nM range). Some specific examples for comparison⁴⁻⁷:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t-FEtuH2KlH2dTfRU1tjNA.png" /></figure><p>The designed binder shows 5.89 µM affinity, roughly 20–100x weaker than natural transcription factors (50 pM to ~300 nM, see table). The paper doesn’t address whether this affinity is sufficient for any application. Micromolar affinity might work for certain applications (e.g., I found a paper on aptamer-based biosensors that detect environmental analytes at these affinities⁸), but transcription regulation or genome editing require nanomolar-level affinities.</p><h4><strong>Epitope coverage not validated</strong></h4><p>The paper uses “motif contacting the DNA” as a constraint during rescaffolding but doesn’t map which residues or which DNA bases are being contacted. The AF3 model shows specific contacts, but these predictions need experimental validation, especially given that AF3 confidence metrics don’t correlate with experimental accuracy for all protein-DNA interfaces⁹.</p><h3>Issue #2: Enzyme Validation Lacks Proper Controls</h3><p>The cysteine hydrolase experiment tested 190 designs and found 35 with multi-turnover activity (18% hit rate). The best design, C6, has kcat/Km = 3557 M⁻¹s⁻¹.</p><h4><strong>No negative control library</strong></h4><p>The paper doesn’t test any negative controls. RFdiffusion2 tested H→A catalytically dead mutants for the most active enzyme design³. Those controls are absent here.</p><p>Standard controls for computational enzyme design:</p><ul><li><strong>Geometry-scrambled active-site controls</strong> using the same scaffold and residue composition but disrupted catalytic geometry</li><li><strong>Catalytically inactive point mutants</strong> (e.g., H→A, C→A) for each active design to separate binding from catalysis</li><li><strong>Scaffold-matched non-catalytic variants</strong> that preserve binding but disrupt key catalytic interactions</li></ul><p>Without these controls, we can’t tell whether:</p><ol><li>The 18% hit rate reflects design quality vs. assay permissiveness</li><li>The activity comes from the designed geometry vs. promiscuous background hydrolysis</li><li>The scaffolds alone (without designed active sites) might show activity</li></ol><p>This is particularly important because the substrate (4-methylumbelliferyl phenyl acetate) is reactive and can undergo non-enzymatic hydrolysis. The paper shows background-subtracted data but doesn’t report the background rate or show that catalytically dead mutants lose activity.</p><h4><strong>Limited kinetic characterization</strong></h4><p>The cysteine hydrolase C6 shows catalytic efficiency (kcat/Km = 3557 M⁻¹s⁻¹), with Michaelis-Menten analysis provided in Figure 4. However, the paper doesn’t report:</p><ul><li>Substrate specificity beyond 4MU-PhAc (does it work on related esters or only this specific substrate?)</li><li>Whether catalytic-dead mutants retain activity</li><li>pH dependence (critical for understanding Cys-His-Asp triad mechanism where protonation states drive catalysis)</li><li>Temperature stability</li></ul><p>RFdiffusion2 reported uncatalyzed rate comparisons and catalytically dead controls for the best design (ZETA_1).</p><h4><strong>Success rate comparison unclear</strong></h4><p>The paper states the best RFdiffusion3 design “exceeds previous designs for the same reaction” but the comparison isn’t straightforward:</p><ul><li>RFdiffusion2: Designed a hydrolase with a Cys-His-Asn catalytic triad, got kcat/Km = 248 M⁻¹s⁻¹ (best design)²</li><li>Earlier Rosetta design: Designed a hydrolase with a Cys-His dyad, got kcat/Km = 405 M⁻¹s⁻¹ (best design)¹⁰</li></ul><p>RFdiffusion3’s kcat/Km = 3557 M⁻¹s⁻¹ is ~14x better than RFdiffusion2’s best and ~9x better than the Rosetta design. But the assays used different substrates:</p><ul><li>RFdiffusion3 used 4-methylumbelliferyl phenyl acetate (4MU–PhAc)</li><li>RFdiffusion2 paper used 4MU-Butyrate²</li><li>2012 Rosetta paper used coumarin ester¹⁰</li></ul><p><strong>These substrates aren’t interchangeable.</strong> They differ in leaving group stability, size, and electronics; all of which affect both Km (binding) and kcat (turnover). A kcat/Km measured on 4MU-PhAc doesn’t directly translate to what you’d measure on 4MU-Butyrate, even with the same enzyme. The only way to know if RFdiffusion3’s enzyme is actually better is to test all three designs on the same substrate under identical conditions.”</p><h3>Issue #3: DNA Shape Prediction Not Validated</h3><p>RFdiffusion3 can predict DNA shape from sequence while designing binding proteins, achieving similar success rates with flexible vs. fixed DNA (8.67% vs. 6.5% for monomers, 6.67% vs. 5.5% for dimers).</p><p>The paper generates protein-DNA complexes by providing DNA sequence and letting the model predict conformation. But DNA shape prediction tested on only 3 structures (7RTE, 7N5U, 7M5W). Given that DNA shape varies substantially with sequence (A-tracts bend DNA, GC vs. AT steps have different geometry, and proteins often distort DNA from B-form¹¹⁻²¹), this small test set limits confidence in the model’s ability to predict diverse DNA shapes.</p><p>The paper experimentally validates one DNA-binding design (DBRFD3) with flow cytometry, measuring EC50 = 5.89 ± 2.15 µM against the target sequence. This demonstrates binding, but key validation is still missing:</p><ul><li>Testing additional designs from the successful set</li><li>Determine structures of designed complexes (cryo-EM or crystallography) to verify predicted geometry</li><li>Measure binding specificity (does it bind the correct sequence vs. mismatches)</li></ul><h3>Issue #4: Missing Baseline and Validation for Hydrogen Bond Conditioning</h3><p>RFdiffusion3 can specify which atoms should act as hydrogen bond donors or acceptors during diffusion. The paper reports this increases hydrogen bond formation from 26.67% to 32.67% (small molecules) or 36.67% with classifier-free guidance. For hydrogen bonds to DNA bases, the improvement is more modest: 11% vs. 11.3% vs. 12.5%.</p><h4><strong>No validation that designed H-bonds actually form</strong></h4><p>These percentages measure how often LigandMPNN sequences maintain donor/acceptor properties at positions flagged by RFdiffusion3. But the paper doesn’t validate that these residues actually form hydrogen bonds with the target:</p><ul><li>No mutagenesis studies showing H-bond residues are required for binding</li><li>No crystal structures showing hydrogen bond geometry</li><li>No hydrogen-deuterium exchange or other spectroscopic confirmation</li></ul><p>The success metric is “does LigandMPNN keep the charge properties at these positions” not “do these positions make hydrogen bonds in the actual complex.”</p><h4><strong>Baseline H-bond rate not contextualized</strong></h4><p>Is 26.67% baseline H-bonds low? The paper doesn’t compare to:</p><ul><li>Native protein-ligand interfaces from PDB (typical H-bond frequencies)</li><li>Previous computational designs</li><li>Negative controls (unrelated protein-ligand pairs)</li></ul><p>Without context, we can’t tell if the unconditioned model already does well or if the conditioning provides substantial improvement.</p><h3>Issue #5: Experimental Selection Criteria Not Disclosed</h3><p>The paper reports success rates for multiple design tasks:</p><ul><li>DNA binders: 8.67% monomeric, 6.67% dimeric</li><li>Small molecules: 10–42% depending on ligand</li><li>Protein binders: varies by target (Figure 3a)</li><li>Enzymes: 18% multi-turnover rate (35/190)</li></ul><p>However, it doesn’t explain the selection criteria for <strong>which computationally successful designs</strong> were chosen for experimental validation. For example, why DBRFD3 was selected from the DNA binders that passed AF3 filters, or why C6 was chosen from the cysteine hydrolases. Without knowing whether experimental candidates were randomly sampled or selected based on additional criteria (predicted confidence, structural diversity, ease of synthesis), it’s unclear whether reported experimental success rates reflect true hit rates or selection bias.</p><h3>What Gets Validated vs. What Gets Claimed</h3><p>RFdiffusion3’s core advance is atom-level conditioning: specifying which atoms donate/accept hydrogen bonds, burial states, center-of-mass positions, and other atomic constraints. This is technically impressive and the speed improvements (10x faster) matter for practical use.</p><p>The paper’s validation doesn’t match the claimed capabilities or the group’s earlier standards:</p><p><strong>Experimentally validated:</strong></p><ul><li>DNA binding to one specific sequence (1 hit from 5 tested, µM affinity)</li><li>Cysteine hydrolase activity (35 hits from 190 tested, best kcat/Km = 3557 M⁻¹s⁻¹)</li></ul><p><strong>Computationally benchmarked (no experimental tests):</strong></p><ul><li>Protein-protein interfaces (5 targets, in silico only)</li><li>Small molecule binders (4 ligands, in silico only)</li></ul><p><strong>Claimed but not validated:</strong></p><ul><li>Sequence-specific DNA recognition (no specificity tests)</li><li>DNA shape prediction accuracy (tested on only 3 structures)</li><li>Small molecule binding modes (no experimental structures)</li><li>Hydrogen bond formation (no mutagenesis or structural validation)</li></ul><p>Compare this to RFdiffusion1: experimental binders for 5/5 targets with nM affinities, cryo-EM structures confirming atomic accuracy, and specificity controls. Or RFdiffusion2 metallohydrolases: catalytically-dead mutants for the best design, and crystal structure validation. The experimental validation here is limited to two applications (DNA binding and enzymes), both with performance below native proteins (µM vs. nM binding for natural TFs; 10³ vs. 10⁶ M⁻¹s⁻¹ for natural hydrolases) though representing improvements over previous computational designs.</p><h3>Conclusion</h3><p>RFdiffusion3 advances protein design by enabling atom-level conditioning at scale. The architectural improvements deliver practical 10x speedups over RFdiffusion2, and the breadth of applications demonstrated (from DNA binding to enzyme design to small molecule recognition) shows the method’s versatility.</p><p>The experimental validation focuses on two applications: DNA binding (1 design tested) and cysteine hydrolases (190 tested, 35 active). Given the group’s track record of rigorous validation in RFdiffusion1 (cryo-EM structures, nM binders, negative controls) and RFdiffusion2 metallohydrolases (crystal structures, mutagenesis, full kinetics), deeper experimental characterization would help establish how reliably these computational predictions translate to functional proteins.</p><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><h4>References:</h4><ol><li>Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., Wicky, B. I. M., Hanikel, N., Pellock, S. J., Courbet, A., Sheffler, W., Wang, J., Venkatesh, P., Sappington, I., Vázquez Torres, S., Lauko, A., De Bortoli, V., Mathieu, E., Ovchinnikov, S., Barzilay, R., Jaakkola, T. S., DiMaio, F., Baek, M., &amp; Baker, D. (2023). <em>De novo design of protein structure and function with RFdiffusion</em>. <em>Nature, 620</em>(7976), 1089–1100. <a href="https://doi.org/10.1038/s41586-023-06415-8">https://doi.org/10.1038/s41586-023-06415-8</a></li><li>Ahern, W., Yim, J., Tischer, D., Salike, S., Woodbury, S. M., Kim, D., Kalvet, I., Kipnis, Y., Coventry, B., Altae-Tran, H. R., Bauer, M. S., Barzilay, R., Jaakkola, T. S., Krishna, R., &amp; Baker, D. (2025). <em>Atom-level enzyme active site scaffolding using RFdiffusion2</em> (bioRxiv Preprint, Version 2). bioRxiv. <a href="https://doi.org/10.1101/2025.04.09.648075">https://doi.org/10.1101/2025.04.09.648075</a> &amp; <a href="https://doi.org/10.1038/s41592-025-02975-x">https://doi.org/10.1038/s41592-025-02975-x</a></li><li>Kim, D., Woodbury, S. M., Ahern, W., Tischer, D., Kang, A., Joyce, E., Bera, A. K., Hanikel, N., Salike, S., Krishna, R., Yim, J., Pellock, S. J., Lauko, A., Kalvet, I., Hilvert, D., &amp; Baker, D. (2025). <em>Computational design of metallohydrolases</em>. <em>Nature, 649</em>(8095), 246–253. <a href="https://doi.org/10.1038/s41586-025-09746-w">https://doi.org/10.1038/s41586-025-09746-w</a></li><li>Jung, C., Bandilla, P., von Reutern, M., Schnepf, M., Rieder, S., Unnerstall, U., &amp; Gaul, U. (2018). <em>True equilibrium measurement of transcription factor-DNA binding affinities using automated polarization microscopy</em>. <em>Nature Communications, 9</em>, 1605. <a href="https://doi.org/10.1038/s41467-018-03977-4">https://doi.org/10.1038/s41467-018-03977-4</a></li><li>Bergqvist, S., Alverdi, V., Mengel, B., Hoffmann, A., Ghosh, G., &amp; Komives, E. A. (2009). <em>Kinetic enhancement of NF-κB·DNA dissociation by IκBα</em>. <em>Proceedings of the National Academy of Sciences of the United States of America, 106</em>(46), 19328–19333. <a href="https://doi.org/10.1073/pnas.0908797106">https://doi.org/10.1073/pnas.0908797106</a></li><li>Li, T., Shahabi, S., Biswas, T., Tsodikov, O. V., Pan, W., Huang, D. B., Wang, V. Y. F., Wang, Y., &amp; Ghosh, G. (2024). <em>Transient interactions modulate the affinity of NF-κB transcription factors for DNA</em>. <em>Proceedings of the National Academy of Sciences of the United States of America, 121</em>(23), Article e2405555121. <a href="https://doi.org/10.1073/pnas.2405555121">https://doi.org/10.1073/pnas.2405555121</a></li><li>Veprintsev, D. B., &amp; Fersht, A. R. (2008). Algorithm for prediction of tumour suppressor p53 affinity for binding sites in DNA. <em>Nucleic Acids Research, 36</em>(5), 1589–1598. <a href="https://doi.org/10.1093/nar/gkm1040">https://doi.org/10.1093/nar/gkm1040</a></li><li>McConnell, E. M., Nguyen, J., &amp; Li, Y. (2020). <em>Aptamer-based biosensors for environmental monitoring</em>. Frontiers in Chemistry, 8, 434. <a href="https://doi.org/10.3389/fchem.2020.00434">https://doi.org/10.3389/fchem.2020.00434</a></li><li>Wee, J., &amp; Wei, G.-W. (2024). <em>Benchmarking AlphaFold3’s protein-protein complex accuracy and machine learning prediction reliability for binding free energy changes upon mutation</em> (arXiv:2406.03979v1). arXiv. <a href="https://doi.org/10.48550/arXiv.2406.03979">https://doi.org/10.48550/arXiv.2406.03979</a></li><li>Richter, F., Blomberg, R., Khare, S. D., Kiss, G., Kuzin, A. P., Smith, A. J. T., Gallaher, J., Pianowski, Z., Helgeson, R. C., Grjasnow, A., Xiao, R., Seetharaman, J., Su, M., Vorobiev, S., Lew, S., Forouhar, F., Kornhaber, G. J., Hunt, J. F., Montelione, G. T., Tong, L., Houk, K. N., Hilvert, D., &amp; Baker, D. (2012). <em>Computational design of catalytic dyads and oxyanion holes for ester hydrolysis</em>. <em>Journal of the American Chemical Society, 134</em>(39), 16197–16206. <a href="https://doi.org/10.1021/ja3037367">https://doi.org/10.1021/ja3037367</a></li><li>Oguey, C., Foloppe, N., &amp; Hartmann, B. (2010). <em>Understanding the sequence-dependence of DNA groove dimensions: Implications for DNA interactions</em>. <em>PLoS ONE, 5</em>(12), e15931. <a href="https://doi.org/10.1371/journal.pone.0015931">https://doi.org/10.1371/journal.pone.0015931</a></li><li>da Rosa, G., Grille, L., Calzada, V., Ahmad, K., Arcon, J. P., Battistini, F., Bayarri, G., Bishop, T., Carloni, P., Cheatham, T. E. III, Collepardo-Guevara, R., Czub, J., Espinosa, J. R., Galindo-Murillo, R., Harris, S. A., Hospital, A., Laughton, C., Maddocks, J. H., Noy, A., Orozco, M., Pasi, M., Pérez, A., Petkevičiūtė-Gerlach, D., Sharma, R., Sun, R., &amp; Dans, P. D. (2021). <em>Sequence-dependent structural properties of B-DNA: What have we learned in 40 years?</em> <em>Biophysical Reviews, 13</em>(6), 995–1005. <a href="https://doi.org/10.1007/s12551-021-00893-8">https://doi.org/10.1007/s12551-021-00893-8</a></li><li>Slavníková, P., Cuker, M., Matoušková, E., Čmelo, I., Zgarbová, M., Jurečka, P., &amp; Lankaš, F. (2025). Sequence-dependent shape and stiffness of DNA and RNA double helices: Hexanucleotide scale and beyond. <em>Journal of Chemical Information and Modeling, 65</em>(17), 9208–9229. <a href="https://doi.org/10.1021/acs.jcim.5c00576">https://doi.org/10.1021/acs.jcim.5c00576</a></li><li>Gardasevic, T., &amp; Noy, A. (2024). <em>The impact of sequence periodicity on DNA mechanics: Investigating the origin of A-tract’s curvature</em>. <em>Nanoscale, 16</em>(39), 18410–18420. <a href="https://doi.org/10.1039/D4NR02571G">https://doi.org/10.1039/D4NR02571G</a></li><li>Hizver, J., Rozenberg, H., Frolow, F., Rabinovich, D., &amp; Shakked, Z. (2001). DNA bending by an adenine–thymine tract and its role in gene regulation. <em>Proceedings of the National Academy of Sciences of the United States of America, 98</em>(15), 8490–8495. <a href="https://doi.org/10.1073/pnas.151247298">https://doi.org/10.1073/pnas.151247298</a></li><li>Bravo, J., Li, Z., Speck, N. A., &amp; Warren, A. J. (2001). <em>The leukemia-associated AML1 (Runx1)–CBFβ complex functions as a DNA-induced molecular clamp</em>. <em>Nature Structural Biology, 8</em>, 371–378. <a href="https://doi.org/10.1038/86264">https://doi.org/10.1038/86264</a></li><li>Young, R. T., Czapla, L., Wefers, Z. O., Cohen, B. M., &amp; Olson, W. K. (2022). <em>Revisiting DNA sequence-dependent deformability in high-resolution structures: Effects of flanking base pairs on dinucleotide morphology and global chain configuration</em>. <em>Life, 12</em>(5), 759. <a href="https://doi.org/10.3390/life12050759">https://doi.org/10.3390/life12050759</a></li><li>Jones, S., van Heyningen, P., Berman, H. M., &amp; Thornton, J. M. (1999). <em>Protein-DNA interactions: A structural analysis</em>. <em>Journal of Molecular Biology, 287</em>(5), 877–896. <a href="https://doi.org/10.1006/jmbi.1999.2659">https://doi.org/10.1006/jmbi.1999.2659</a></li><li>Stella, S., Cascio, D., &amp; Johnson, R. C. (2010). <em>The shape of the DNA minor groove directs binding by the DNA-bending protein Fis</em>. <em>Genes &amp; Development, 24</em>, 814–826. <a href="https://doi.org/10.1101/gad.1900610">https://doi.org/10.1101/gad.1900610</a></li><li>Biswas, A., &amp; Basu, A. (2023). <em>The impact of the sequence-dependent physical properties of DNA on chromatin dynamics</em>. <em>Current Opinion in Structural Biology, 83</em>, 102698. <a href="https://doi.org/10.1016/j.sbi.2023.102698">https://doi.org/10.1016/j.sbi.2023.102698</a></li><li>Hsu, C., Verkuil, R., Liu, J., Lin, Z., Hie, B., Sercu, T., Lerer, A., &amp; Rives, A. (2022). <em>Learning inverse folding from millions of predicted structures</em>. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, &amp; S. Sabato (Eds.), <em>Proceedings of the 39th International Conference on Machine Learning</em> (Vol. 162, pp. 8946–8970). Proceedings of Machine Learning Research. <a href="https://proceedings.mlr.press/v162/hsu22a.html?utm_source=chatgpt.com">https://proceedings.mlr.press/v162/hsu22a.html</a></li><li>Yonetani, Y., &amp; Kono, H. (2009). <em>Sequence dependencies of DNA deformability and hydration in the minor groove</em>. <em>Biophysical Journal, 97</em>(4), 1138–1147. <a href="https://doi.org/10.1016/j.bpj.2009.05.049">https://doi.org/10.1016/j.bpj.2009.05.049</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=03c2208785d4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Low Immunogenicity in Human Panels? Examining Latent-X2’s Evidence]]></title>
            <link>https://medium.com/@enginyapici/low-immunogenicity-in-human-panels-examining-latent-x2s-evidence-9c540b8f49ba?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/9c540b8f49ba</guid>
            <category><![CDATA[antibody-engineering]]></category>
            <category><![CDATA[protein-engineering]]></category>
            <category><![CDATA[biotechnology]]></category>
            <category><![CDATA[protein-design]]></category>
            <category><![CDATA[antibody]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 30 Dec 2025 16:11:25 GMT</pubDate>
            <atom:updated>2025-12-30T18:20:56.839Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://www.latentlabs.com/">Latent Labs</a> released preprint data on <a href="https://www.latentlabs.com/latent-x2/">Latent-X2</a>, their generative model for antibody and macrocycle design. <strong>This is the first study to report immunogenicity data for AI-generated antibodies, a critical milestone, since immunogenicity is a leading cause of clinical failure.</strong> The paper’s lead claim is “drug-like antibodies with low immunogenicity in human panels.”</p><p><strong>But the immunogenicity evidence doesn’t support that claim.</strong> They tested only 4 VHH binders from a single target (TNFL9) in Fc-fusion format, not the naked VHH domains they designed. The 10-donor panel is heavily biased: 60% of donors share the same HLA B44 supertype and 40% share B08, meaning you’re testing ~7–8 immune backgrounds, not 10 independent ones. The assay lacks appropriate positive controls (no bococizumab, no KLH), uses indirect detection methods (bulk cytokines and ATP instead of T-cell activation markers), and has no predefined statistical framework to define what counts as an immunogenic response.</p><p><strong>Beyond immunogenicity, the binding validation has gaps.</strong> The paper reports picomolar affinities for two HDAC8 binders (26–41 pM), but both sensorgrams show baseline drift during dissociation, a technical artifact that prevents accurate KD determination. There’s no epitope mapping for any construct, no competition binding data, no functional validation beyond binding.</p><p>In this post, I analyze both the immunogenicity assay quality and the overall reporting of tested binders.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bdHsQ158abqrGqmD8bsdOA.png" /><figcaption>Binding validation for computationally designed VHHs across 7 targets. Blue structures are VHH designs, red structures are scFv designs. Surface plasmon resonance (SPR) and biolayer interferometry (BLI) sensorgrams show concentration-dependent binding with fitted KD values. Black dashed lines show kinetic fits. Reported affinities span picomolar (HDAC8 binders at 26–41 pM) to micromolar (ONCM binder at 3.26 μM), with most VHH designs in the 2.75–797 nM range. Several sensorgrams show technical issues: TNFL9_10 displays biphasic binding; ONCM_3 shows potential biphasic behavior; multiple targets (MMP2, AHSP, LEP) exhibit incomplete dissociation; both HDAC8 constructs show baseline drift during dissociation; 1433B_1 and 1433B_6 report identical KD values (4.27 nM) despite different kinetic profiles.</figcaption></figure><h3>How The Model Works</h3><p>Latent-X2 is an all-atom generative model that jointly designs sequence and structure. The paper doesn’t disclose architectural details.</p><p><strong>The model takes 3 inputs:</strong></p><ul><li>Target protein structure (only the backbone atoms)</li><li>Epitope specification</li><li>Optional antibody framework with desired CDR lengths</li></ul><p><strong>And it outputs three things:</strong></p><ul><li>Complete binder sequence</li><li>All-atom 3D structure of the binder</li><li>The bound complex with the target</li></ul><p><strong>Multi-modality:</strong> The model can generate different types of binders from the same architecture:</p><ul><li>VHH (nanobodies): Single-domain antibodies</li><li>scFv: Two-domain antibodies with VH and VL chains connected by a linker</li><li>Macrocycles: Cyclic peptides, 6–18 residues</li><li>Mini-binders</li></ul><p>You specify which modality you want during generation. For antibodies, you provide the framework and desired CDR lengths. For macrocycles, you specify the peptide length. Maximum input length is 512 residues, target + binder together.</p><h3>Results</h3><p>Against 20 targets, they tested 4–24 designs per modality (i.e., VHH, scFv, macrocycle) per target.</p><ul><li>~55% target-level reported success rate (11/20 targets; 9 antibody targets + 2 macrocycle targets).</li><li>No success rate reported per modality per target (i.e., we don’t know how many designs were generated &amp; tested).</li><li>47% of binders met all four developability criteria without optimization (monomericity, hydrophobicity, thermostability, polyreactivity)</li></ul><h4>Methods used:</h4><ul><li>SPR or BLI for binding kinetics and affinity</li><li>Baculavirus particle (BVP) ELISA for polyreactivity</li><li>NanoDSF for thermal stability (Tm)</li><li>HIC-HPLC for hydrophobicity (used lirilumab and tremelimumab as positive and negative controls, respectively per <a href="https://www.pnas.org/doi/10.1073/pnas.1616408114">Jain et al. 2017</a>)</li><li>SEC-HPLC for monomericity</li><li>Cell-based assay with human PBMCs for <em>ex vivo</em> immunogenicity</li></ul><h4>Binders:</h4><ul><li><a href="https://platform.latentlabs.com/design/6baf4691-9816-4bfe-8db9-c40d4e019d9c/output/demo">HDAC8</a> (26.2 pM, 41.5 pM, both scFvs)</li><li><a href="https://platform.latentlabs.com/design/714cd785-40f8-4867-bb1f-0bf91c4f861a/output/demo">1433B</a> (2.75 nM VHH; 4.27 nM VHH*; 4.27 nM scFv*)</li><li><a href="https://platform.latentlabs.com/design/b3e85ab7-7a4a-4b7c-a5b5-687a27763390/output/demo">LEP</a> (44.8 nM VHH)**</li><li><a href="https://platform.latentlabs.com/design/a2d4c26d-d09c-4b80-9d65-7a9f11b0b808/output/demo">BHRF1</a> (27.0 nM VHH)**</li><li><a href="https://platform.latentlabs.com/design/096ae975-21bd-45f1-a571-e0fc79095296/output/demo">TNFL9</a> (31.1 nM, 45 nM***, 49.7 nM***, all VHHs. 5/24 reported success rate.)</li><li><a href="https://platform.latentlabs.com/design/5903021f-1dbb-4319-b2ac-d225f52db745/output/demo">MMP2</a> (77 nM, 555 nM, both VHHs)</li><li><a href="https://platform.latentlabs.com/design/97a4e43c-10f7-454b-9348-66fad88fba58/output/demo">SAE1</a> (165 nM scFv)</li><li><a href="https://platform.latentlabs.com/design/9415207c-283d-4e18-aa7e-629f6a79425d/output/demo">AHSP</a> (797 nM)</li><li><a href="https://platform.latentlabs.com/design/38b75076-cc6e-440c-9918-2f6125766cf2/output/demo">ONCM</a> (3.26 µM)</li><li>K-Ras(G12D) (5.53 µM, 23.90 µM, both macrocycles. 8/10 reported success rate.)</li><li>PHD2 (1.54 nM, 4.70 nM, both macrocycles. 9/10 reported success rate.)</li></ul><p>Targets failed: 1433E, CD98, IL-20, IL-3, PD-L1, PRL, SC2RBD, SOMA, UBE2B</p><p><em>* There might be an error in Figure 2 for the KD values calculated for LX2_VHH_1433B_1 and LX2_scFv_1433B_6. They both have the same KD value with different kinetics profiles.</em></p><p><em>** There is a discrepancy between the KD values reported in the paper and the values on their website (</em><a href="https://platform.latentlabs.com/design/b3e85ab7-7a4a-4b7c-a5b5-687a27763390/output/demo"><em>link</em></a><em> and </em><a href="https://platform.latentlabs.com/design/2bb85ea5-9124-4128-98cc-424189262a68/output/demo"><em>link</em></a><em>). Additionally, there are 2 more designs listed per target on the website that don’t appear in the paper.</em></p><p><em>*** These two designs are not listed on the website. Only the first binder (31.1 nM) is listed.</em></p><p>They report antibody binders for 9 targets but tested immunogenicity for only 4 binders from 1 target (TNFL9). They confirmed 5 TNFL9 binders total but tested 4 for immunogenicity. Why exclude 1? What about VHHs to MMP2, LEP, 1433B, BHRF1, AHSP, ONCM? What about the scFvs including their best binder?</p><p><strong>For TNFL9 immunogenicity:</strong> Four VHH binders showed no detectable immunogenic response across 10 healthy human donors in T-cell proliferation and cytokine release assays. But the paper reports binding data for only 3 of these. We don’t know how and why they picked those specific 4 VHHs for immunogenicity testing.</p><p><strong>For macrocycle success rates:</strong> K-Ras(G12D) and PHD2 showed 80–90% hit rates from 10 designs per target, matching or exceeding trillion-scale mRNA display screens. But they only report KD values for 2 binders per target. That’s 13 confirmed binders with missing affinity data.</p><h3>Reporting, Transparency, and Validation Gaps</h3><h4>1. Picomolar affinity claim based on unoptimized BLI experiment</h4><p>Authors report picomolar affinities for two HDAC8-binding scFvs (26.2 pM and 41.5 pM) based on BLI measurements. However, both sensorgrams show baseline drift during the dissociation phase, which prevents accurate determination of the dissociation rate constant (koff). Since KD = koff/kon, compromised dissociation kinetics directly undermine the affinity calculation.</p><p>Baseline drift during dissociation typically indicates:</p><ul><li>Non-specific aggregation on the sensor surface</li><li>Avidity effects (especially problematic for scFv constructs, which can form dimers)</li><li>Rebinding due to insufficient washing or mass transport limitations</li><li>Assay conditions not optimized for the interaction</li></ul><p>Picomolar binding claims require rigorous validation. When SPR/BLI data show artifacts, standard practice is to:</p><ol><li><strong>Optimize assay conditions</strong> (buffer composition, flow rate, surface density, regeneration conditions)</li><li><strong>Use orthogonal methods</strong> (e.g., KinExA)</li><li><strong>Demonstrate functional consequences</strong> (sub-picomolar binders should show potent inhibition at low nM concentrations)</li></ol><p>The preprint provides none of these. The HDAC8 constructs likely bind tightly (the sensorgrams show clear concentration-dependent responses) but <strong>the actual KD values cannot be determined from these data</strong>. Without optimized dissociation kinetics or orthogonal validation, the picomolar claims are not supported by the data shown. These affinities should be reported as “low nanomolar or better” until proper validation confirms sub-nanomolar binding.</p><h4><strong>2. No success rates reported per modality per target</strong></h4><p>The paper states they tested “4–24 designs per modality per target” but only reports exact counts for 3 of 11 successful targets: TNFL9 (5/24 VHH = 21%), PHD2 (9/10 macrocycles = 90%), and K-Ras(G12D) (8/10 macrocycles = 80%). For the other 8 targets, we don’t know how many designs were tested per modality. I expected to see “X designs tested, Y binders confirmed” per target per modality.</p><h4><strong>3. Missing affinity data for confirmed binders</strong></h4><p>PHD2 had 9 confirmed binders but only 2 KD values are reported. K-Ras(G12D) had 8 confirmed binders but only 2 KD values are reported. That’s 13 missing KD values. They ran SPR or BLI to confirm binding, so the data exists. Why report only the best 2 per target? Are the other binders weak? Did they fail developability?</p><h4><strong>4. Paper-website discrepancies</strong></h4><p>Multiple targets show inconsistencies between the preprint and their website data:</p><p><strong>LEP:</strong> Different KD values reported in paper versus website. The website also lists 2 additional designs that don’t appear in the paper.</p><p><strong>BHRF1:</strong> Different KD values reported in paper versus website. The website also lists 2 additional designs that don’t appear in the paper.</p><p><strong>TNFL9:</strong> This one is especially problematic. The paper reports 5 confirmed binders (31.1 nM, 45.0 nM, 49.7 nM, plus two others). They tested 4 of these for immunogenicity. But the website only lists 1 binder: the 31.1 nM design.</p><p>Why do some binders appear in the paper but not the website? Why do some appear on the website but not in the paper?</p><h4><strong>5. Unexplained selective reporting for immunogenicity testing</strong></h4><p>TNFL9 had 5 confirmed VHH binders. They tested 4 for immunogenicity and excluded 1 without explanation. Were the 4 selected randomly? Best binders? Most developable? Lowest predicted immunogenicity? If you’re claiming “no immunogenicity detected,” excluding 20% of your binders without explanation undermines that claim.</p><h4><strong>6. No failure mode analysis</strong></h4><p>9 targets failed (1433E, CD98, IL-20, IL-3, PD-L1, PRL, SC2RBD, SOMA, UBE2B). Zero information on which modalities were tested, how many designs per modality, or why they failed. You cannot assess model limitations without understanding failures.</p><h4>7. No Epitope Validation</h4><p>They claim the model generates binders that “engage the specified epitope” and that “binding specificity was assessed by alanine mutagenesis of key CDRH3 residues.” They mutated 3 positions in CDRH3, saw loss of binding in all 4 VHHs, and concluded this confirms “target recognition being driven by the designed epitope interactions.”</p><p>This doesn’t validate epitope specificity. Mutating CDRH3 and losing binding just shows those residues are important for binding. It doesn’t tell you where on the target you’re binding. You need actual epitope mapping (hydrogen-deuterium exchange, cross-blocking experiments, or co-crystal structures) to figure out if you are hitting the intended epitope or binding somewhere else that still requires those CDRH3 residues.</p><h4><strong>8. Undisclosed expression system selection criteria</strong></h4><p>The paper uses two different expression systems (mammalian, CHO cells, and cell-free) but provides no rationale for why specific targets were assigned to each system. From the methods (Section D.1):</p><ul><li><strong>CHO expression (14 targets)</strong>: 1433B, 1433E, AHSP, CD98, HDAC8, IL-20, IL-3, LEP, MMP2, ONCM, PRL, SOMA, TNFL9, UBE2B</li><li><strong>Cell-free expression (4 targets)</strong>: BHRF1, PD-L1, SAE1, SC2RBD</li></ul><p>Expression system affects protein folding, post-translational modifications, yield, and ultimately whether you can even test a design. Cell-free expression is sometimes used as a rescue strategy when designs fail in mammalian cells due to toxicity, poor folding, or low expression. If that happened here, it means some designs failed very early with critical developability/manufacturability criteria. A single sentence explaining the decision criteria would resolve this issue.</p><h3>Immunogenicity Assay Quality Gaps</h3><h4><strong>9. Single-target, single-format assessment</strong></h4><p>Only TNFL9 was tested for immunogenicity. That’s 1 of 9 successful antibody targets. Only the VHH format was tested, not scFv (which includes their best binder at 26.2 pM). No explanation for why TNFL9 was selected. We cannot assess whether this is typical performance or a cherry-picked low-immunogenicity example.</p><h4><strong>10. Wrong format tested</strong></h4><p>All VHHs were expressed as Fc-fusion constructs in pcDNA3.4 vectors with a human IgG1 Fc tag (hinge-CH2-CH3). This means they tested Fc-VHH chimeras, not naked VHH domains. The Fc region introduces additional biology through FcRn binding, Fcγ receptor–mediated APC uptake, immune complex formation, and innate effector signaling. These processes alter antigen exposure, processing, and presentation compared to a naked VHH. As a result, an Fc-fused VHH represents a different molecular format with a distinct immunogenic risk profile. Testing Fc-fusion constructs does not directly support claims about intrinsic VHH immunogenicity, only about that specific Fc-linked format.</p><p><strong>Control format ambiquity:</strong> The paper doesn’t specify whether they reformatted caplacizumab to match their test constructs. Caplacizumab is approved as a bivalent VHH (two VHH domains linked together), not an Fc-fusion. For a valid comparison, they should have either: (1) cloned and expressed caplacizumab in-house with the same Fc tag used for their designs, or (2) tested their VHHs in bivalent format to match caplacizumab’s approved structure. We don’t know which (if either) they did.</p><h4>11. HLA supertype imbalance</h4><p>The 10-donor panel has some imbalances. HLA alleles cluster into supertypes based on shared peptide-binding specificities. Donors within the same supertype present similar epitope repertoires and can show correlated immunogenic responses.</p><p>Analyzing the donor panel by HLA supertype shows both overrepresentation of certain supertypes and underrepresentation of others:</p><ul><li><strong>B44 supertype dominates (60%)</strong>: B*18:01 (donors 1, 3), B*40:01 (donors 2, 10), B*41:02 (donor 8), B*50:01 (donor 7). B44 alleles prefer acidic residues at P2 (Glu, Asp) and aromatic/hydrophobic C-termini. Having 60% of donors share this specificity creates <strong>redundancy in peptide presentation</strong>. While B44 coverage is important (representing ~20–30% of most populations), this level of overrepresentation reduces the effective diversity of the panel.</li><li><strong>B08 supertype overrepresented (40%)</strong> Four of 10 donors carry B*08:01 (donors 1, 2, 5, 6). B08 supertype alleles have distinct peptide-binding preferences that create a separate peptide-presentation context. This oversampling reduces effective diversity.</li><li><strong>B62 supertype severely underrepresented (10%)</strong> Only 1 donor carries a B62 allele (B*15:02 in donor 4). B62 alleles prefer aliphatic residues at P2 and cover a significant portion of many populations. Single-donor representation provides insufficient coverage for this supertype.</li><li><strong>Other B supertypes with reasonable coverage:<br>- </strong>B07 supertype: 30% (3/10 donors: B*07:02, B*42:01, B*51:01)<br>- B27 supertype: 20% (2/10 donors: B*27:05, B*39:06)<br>- B58 supertype: 30% (3/10 donors: B*57:01, B*58:01, B*58:02)</li><li><strong>HLA-A coverage is better but still clustered</strong>:<br>- A02 supertype: 40% (4/10 donors)<br>- A03 supertype: 30% (3/10 donors)<br>- A01 supertype: 20% (2/10 donors)<br>- A24 supertype: 20% (2/10 donors)</li></ul><p>The effective ’N’ of this study is not 10, because of the imbalance with overrepresentation of B44 (60%) and B08 (40%), and severe underrepresentation of B62 (10%). Functional diversity of the peptide-presentation space is closer to an <strong>N of 7 or 8. </strong>When 60% of donors share the B44 supertype and 40% share B08, negative results cannot reliably exclude immunogenicity across diverse populations. Oversampling of B44 and B08 supertypes combined with B62 underrepresentation increases <strong>Type II error risk</strong> and limits the panel’s ability to detect immunogenicity in underrepresented immune contexts.</p><p><em>Note: I mapped the HLA types using </em><a href="http://www.biomedcentral.com/1471-2172/9/1"><em>Sidney et al. 2008 paper</em></a><em>, based on their supertype classifications. Full mappings can be found in the Appendix, along with my recommended HLA type coverages.</em></p><h4><strong>12. Missing high-immunogenicity positive controls</strong></h4><p>They included PHA-L, anti-CD3, and anti-CD28 as positive controls. These are polyclonal activators. They don’t test the HLA-II/TCR antigen-specific pathway that causes therapeutic antibody immunogenicity. You need protein antigens known to be immunogenic, like bococizumab (48% ADA in patients), HuA33 (73% ADA), or KLH (keyhole limpet hemocyanin, the universal positive control). Without these, you cannot distinguish between “low immunogenic response” and “assay cannot detect responses.”</p><p>Here’s the irony. They do use bococizumab in this paper as the thermostability benchmark. Bococizumab (Tm = 61°C) defines their “acceptable” threshold in the developability criteria section. But bococizumab failed Phase 3 trials due to 48% ADA rates. They cite this failure in reference 17. So they use a clinical failure with massive immunogenicity as a developability standard but don’t use it as an immunogenicity positive control.</p><p>Why not also include KLH as well? It’s highly immunogenic in all donors, tests the full HLA-II/TCR pathway, and is used in vaccine adjuvant studies. If your assay can’t detect KLH immunogenicity, your assay doesn’t work.</p><h4><strong>13. Low assay sensitivity</strong></h4><p>PBMC-based T-cell activation assays have a fundamental sensitivity problem. Antigen-specific CD4+ T cells are rare. Detection depends on antigen dose and sampling enough donors to cover HLA class II diversity. This study used 30 µg/mL protein (maximum) and 10 donors. Those are conservative choices for this assay type. With these parameters, you cannot tell the difference between “no immunogenic response” and “we didn’t sample enough rare T-cell clones or HLA combinations to find a response.” Some groups run these assays with higher protein concentrations and 40+ donors specifically to address this limitation (e.g., <a href="https://doi.org/10.1080/19420862.2021.1898831">Cohen et al. 2021</a>).</p><h4><strong>14. Indirect detection methods</strong></h4><p>They measured T-cell activation indirectly: ATP levels (via CellTiter-Glo) for proliferation and bulk cytokine secretion for activation. PBMCs contain multiple cell types (T-cells, B-cells, monocytes, NK cells) all contributing to these readouts. ATP measures metabolic activity from all cells, not just dividing T-cells. Cytokines like TNF-α and IFN-γ come from monocytes and NK cells, not just T-cells. IL-8 is constitutively produced. These indirect measures mix signals from different cell types and pathways, making it harder to detect antigen-specific T-cell responses. Direct methods use activation markers like CD134 and CD137 to count activated CD4+ T-cells specifically.</p><h4><strong>15. No statistical framework</strong></h4><p>They don’t define what counts as an immunogenic response. No stimulation index calculation, no predefined threshold, no statistical significance testing. Their conclusion of “no detectable immunogenic response” comes from looking at bar graphs in Figure 3. <strong>Without objective criteria, you cannot distinguish between “no response,” “response below detection threshold,” and “assay noise.”</strong> The negative result isn’t falsifiable. You need a predefined decision framework to make quantitative claims about immunogenicity risk.</p><h4><strong>16. No mechanism confirmation</strong></h4><p>No controls to confirm the assay measures HLA-II restricted CD4+ T-cell activation. No HLA blocking experiments. They tested Fc-fusion proteins in PBMC culture. You need to rule out Fc-mediated effects and innate responses. Without these controls, you cannot tell if you’re measuring antigen-specific T-cell responses or something else.</p><h4><strong>17. Technical execution problems</strong></h4><p>They cultured cells for 120 hours. That’s long enough for cell death, overgrowth, and nonspecific activation. They don’t report viability measurements. They froze and thawed supernatants before cytokine analysis. IL-10 and IL-6 are unstable under freeze-thaw. They didn’t deplete CD8+ T-cells, which is needed for CD4+ T-cell specific responses. IL-8 was “above range” for multiple conditions, meaning the assay wasn’t optimized for the concentration range. They used only 2 technical replicates per condition with <strong>no biological replicates</strong>.</p><h3>Conclusion</h3><p>Latent-X2 is the first AI antibody design study to publish immunogenicity data. Immunogenicity assays are not cheap to run and they can get finicky. I am very happy to see it is finally included in an AI antibody design paper. Considering how critical immunogenicity is downstream, I hope to see more teams willing to go the extra mile and include similar assays in their developability panels.</p><p>Here’s what would make this work stronger: Test immunogenicity across multiple targets and formats, not just 4 VHHs from TNFL9. Include bococizumab and KLH as positive controls. Use direct T-cell activation markers and predefined statistical thresholds. Report design counts per modality, all confirmed binder affinities, and failure modes. Make the paper and website datasets consistent.</p><p>The binding work demonstrates functional binders across multiple targets with reasonable developability profiles. The macrocycles match trillion-scale mRNA display screens in hit rate. But the picomolar affinity claims for HDAC8 aren’t supported because both sensorgrams show baseline drift during dissociation that prevents accurate KD determination. Without optimized assays or orthogonal validation, those should be reported as “low nanomolar or better,” not 26–41 pM.</p><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><h3>Appendix</h3><h4>HLA Type Mappings</h4><p>Full map can be found here: <a href="https://docs.google.com/spreadsheets/d/1RD6BHRZtyMg9CMJ9pFZK894NIweUJxRx_dCXstlc2uw/edit?usp=sharing">link</a>.</p><p><strong>HLA-A Supertypes:</strong></p><ul><li><strong>A01</strong>: 5/10 donors (50%): donors 1, 2, 3, 5, 10</li><li><strong>A01/A03</strong>: 1/10 donors (10%): donor 3 (A*30:01)</li><li><strong>A02</strong>: 4/10 donors (40%): donors 2, 5, 6, 7</li><li><strong>A03</strong>: 4/10 donors (40%): donors 4, 8, 9, 10</li><li><strong>A24</strong>: 2/10 donors (20%): donors 4, 7</li><li><strong>A01/A24</strong>: 0/10 donors (0%): ABSENT</li></ul><p><strong>HLA-B Supertypes:</strong></p><ul><li><strong>B07</strong>: 3/10 donors (30%): donors 3, 4, 9</li><li><strong>B08</strong>: 4/10 donors (40%): donors 1, 2, 5, 6</li><li><strong>B27</strong>: 2/10 donors (20%): donors 6, 7</li><li><strong>B44</strong>: 6/10 donors (60%): donors 1, 2, 3, 7, 8, 10</li><li><strong>B58</strong>: 3/10 donors (30%): donors 8, 9, 10</li><li><strong>B62</strong>: 1/10 donor (10%): donor 4</li><li><strong>Unclassified</strong>: 1/10 donor (10%): donor 5 (B*13:01)</li></ul><h4><strong>Acceptable Supertype Coverage</strong></h4><p>For a 10-donor immunogenicity panel to provide adequate population coverage, supertype representation should approximate global HLA frequencies while ensuring all major supertypes have sufficient representation to detect potential epitopes.</p><p><strong>Minimum Acceptable Coverage (10-donor panel):</strong></p><p><strong>HLA-A Supertypes:</strong></p><ul><li>A01: 2 donors (20%)</li><li>A02: 4–5 donors (40–50%)</li><li>A03: 3 donors (30%)</li><li>A24: 2 donors (20%)</li></ul><p><strong>Current panel A coverage is acceptable</strong>, though A02 is slightly underrepresented at 40% (expected ~50%).</p><p><strong>HLA-B Supertypes:</strong></p><ul><li>B07: 3–4 donors (30–40%) → <strong>Current: 3 (30%) ✓</strong></li><li>B08: 2 donors (20%) → <strong>Current: 4 (40%) – 2x overrepresented</strong></li><li>B27: 1–2 donors (10–20%) → <strong>Current: 2 (20%) ✓</strong></li><li>B44: 2–3 donors (20–30%) → <strong>Current: 6 (60%) – 2–3x overrepresented</strong></li><li>B58: 1–2 donors (10–20%) → <strong>Current: 3 (30%) ✓</strong></li><li>B62: 2 donors (20%) → <strong>Current: 1 (10%) – 2x underrepresented</strong></li></ul><p><strong>What the panel should look like:</strong></p><p>To correct these imbalances, the study should:</p><ol><li><strong>Reduce B44 representation from 6 to 2–3 donors</strong> (removing 3–4 donors)</li><li><strong>Reduce B08 representation from 4 to 2 donors</strong> (removing 2 donors)</li><li><strong>Increase B62 representation from 1 to 2 donors</strong> (adding 1 donor)</li></ol><p>This would require replacing 4–5 donors to achieve balanced coverage. Alternatively, <strong>expanding to 20–30 donors</strong> would allow balanced representation across all supertypes without requiring perfect matching to population frequencies, which is impractical with only 10 donors.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9c540b8f49ba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Analyzing BoltzGen: MIT’s Multi-Target Binder Design Model]]></title>
            <link>https://medium.com/@enginyapici/analyzing-boltzgen-mits-multi-target-binder-design-model-ef3229e22af4?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef3229e22af4</guid>
            <category><![CDATA[biotechnology]]></category>
            <category><![CDATA[protein-design]]></category>
            <category><![CDATA[ai-in-healthcare]]></category>
            <category><![CDATA[protein-engineering]]></category>
            <category><![CDATA[antibody]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 16 Dec 2025 16:23:38 GMT</pubDate>
            <atom:updated>2025-12-16T16:31:05.649Z</atom:updated>
            <content:encoded><![CDATA[<p>MIT released <a href="https://www.biorxiv.org/content/10.1101/2025.11.20.689494v1">BoltzGen</a> for computational binder design in collaboration with multiple academic labs and Adaptyv Bio*. The model handles proteins, peptides, nanobodies, and small molecules with varying success:</p><ul><li>Benchmark targets (5 proteins with known binders in training data): 4/5 showed nM binders for both nanobodies and proteins, including pM hits on PDGFR.</li><li>66% hit rate on 9 novel protein targets (&lt;30% sequence identity to anything bound in PDB), testing 15 designs per target. They designed both VHHs and proteins of lengths 80–140 residues. For VHHs, they randomly sampled from 4 different nanobody scaffolds. Both yielded binders against 6 targets out of 9; 7 if you combine both sets of designs. No binders were found for 2 targets for either.</li><li>Designed functional antimicrobial peptide neutralizers: 2/3 targets (melittin, indolicidin) showed nM binding and neutralized antimicrobial activity. Third target (protegrin) bound at µM but also neutralized activity.</li><li>GyrA antimicrobial peptides: 352/1808 designs (19.5%) inhibited <em>E. coli</em> growth &gt;4x, with 54 (3.0%) showing interface-specific activity confirmed by alanine scanning.</li><li>Cyclic peptide binders to RagA:RagC complex: 14/24 tested designs bound (58%), with 8 showing resolved affinities between 80–1100 µM.</li><li>First designs targeting disordered protein region (NPM1): 1/5 designs showed nucleolar co-localization in live cells, though binding site not definitively confirmed.</li><li>Small molecule binding (rucaparib): 5/6 designs bound at 50–150 µM. Weak affinities but demonstrates cross-modality capability.</li><li>Penguinpox and hemagglutinin nanobodies: 1/7 and 7/7 showed yeast display signal, but weak (≤2 µM based on assay limits).</li></ul><p>But there are gaps worth examining.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rPlB3TiPC6Q4JSsEgh12bQ.jpeg" /><figcaption><strong>BoltzGen experimental validation across diverse targets and modalities.</strong> Top left: Cyclic peptide binders to RagC GTPase and RagA:RagC dimer, with antimicrobial peptide neutralizers (melittin, indolicidin, protegrin). Middle left: GyrA antimicrobial peptides showing in-cell <em>E. coli</em> growth inhibition. Top right: Nanobody and protein binders to recently deposited PDB structures (penguinpox, hemagglutinin) and small molecule binding (rucaparib). Bottom: Nanobody and protein binders against benchmark targets with known binders in training data (IL-7RA, PDL1, PDGFR, Insulin, TNF-a). Success rates varied from 66% on novel targets to 80% on benchmark targets, with affinities ranging from picomolar (PDGFR) to micromolar (small molecules).</figcaption></figure><h3>Architecture and Training</h3><p>I’m not an ML expert. I tried my best to understand and summarize what I got from the paper. Here it is:</p><p>BoltzGen uses geometric encoding for designed residues: 14 atoms per position, with residue identity encoded by superposition patterns onto backbone atoms. Three atoms on backbone nitrogen plus four on oxygen means threonine. This lets them train on structure prediction and design simultaneously without discrete/continuous variable mixing.</p><p>The model matches Boltz-2 folding performance. Training used PDB, AlphaFold DB, and Boltz-1 distillation with crop sizes of 768 residues for folding, 512 for design. Multiple task types: folding, binder design, motif scaffolding, unconditional generation.</p><p>The key point: they unified structure prediction and binder design in one model rather than separating them. Whether this architectural choice drives the experimental results or if it’s the filtering and inverse folding doing the heavy lifting, I can’t tell from the data they show.</p><h3>The Pipeline: 60,000 to 15</h3><p>BoltzGen generates 60,000 initial designs per target. These get filtered down to typically 15 for experimental testing. The paper doesn’t report how many pass computational filters before final selection. If only a few dozen survive the filters, then the filters are doing heavy lifting to find the rare good designs. This is important for understanding whether the model generates mostly good designs (and filters just remove obvious failures) or mostly bad designs (and filters work hard to find the few viable ones).</p><p>Their ranking uses worst-rank-across-metrics: design_iiptm, design_ptm, neg_min_design_to_target_pae, hydrogen bonds, salt bridges, buried surface area¹. Metric weights were calibrated on 11,000 validated binders.</p><h3>Epitope Validation Gap</h3><p>Unlike Chai-2, which used systematic mutagenesis and cryo-EM to validate predicted binding sites, BoltzGen doesn’t consistently show designs bind where predicted.</p><p>The NPM1 campaign: they designed peptides targeting NPM1’s disordered region, got one design showing nucleolar co-localization in cells, but explicitly note “this experiment does not definitely confirm that the binding occurs specifically at the disordered region: the main evidence for this comes from the BoltzGen-generated structure.”</p><p>So the evidence it binds the disordered region is that the model said it would.</p><p>For the 9 novel targets with 66% success, binding is confirmed but epitopes aren’t mapped. We don’t know if they bound where predicted. The GyrA work included alanine scan controls (replacing three interface residues with alanine abolished activity for 3–6% of inhibitory designs), which suggests some designs bind as intended. But that’s one campaign out of eight.</p><h3>Inverse Folding Confounds</h3><p>BoltzGen’s design process has two steps: first it generates a 3D protein backbone shape, then it uses BoltzIF (their inverse folding tool) to figure out which amino acid sequence would fold into that shape. BoltzIF performs about as well as ProteinMPNN and SolubleMPNN on standard tests.</p><p>The paper does not isolate the contribution of backbone generation versus inverse folding, making it difficult to attribute experimental success to either step. Is it the backbone generation (BoltzGen’s novel part) or the sequence design (BoltzIF, which is comparable to existing tools)?</p><p>They didn’t test the obvious controls:</p><ul><li>BoltzGen backbones with ProteinMPNN sequences</li><li>RFdiffusion backbones with BoltzIF sequences</li></ul><p>If BoltzGen backbones + ProteinMPNN still gets 66% success, then the backbone generation is doing the work. If RFdiffusion backbones + BoltzIF also gets 66%, then BoltzIF is doing the work and the backbone generator doesn’t matter much.</p><p>There’s indirect evidence BoltzIF might be critical. Early versions of BoltzGen (v0) had a bug that gave 0/7 success on hemagglutinin. The current version gets 7/7.</p><p>The bug: designs would fold correctly when bound to the target, but couldn’t fold into that structure on their own. They needed the target protein to hold them in the right shape, which doesn’t work in real experiments where the binder has to fold first, then bind.</p><p>They fixed this by adding a computational check: does the design fold correctly in isolation, without the target present? If not, reject it.</p><p>The filter catches bad designs that the model still generates. The backbone generator produces designs that can’t fold on their own, and the filter removes them before experimental testing. If the filter is doing this much work, how much credit does the generative model deserve for the 66% success rate?</p><h3>Computational Benchmarks</h3><p>On 110 targets, BoltzGen shows higher design diversity than RFdiffusion/RFdiffusionAA when each designs one binder per target. This indicates BoltzGen uses the target structure more effectively when generating designs, rather than producing similar outputs regardless of what target you give it.</p><p>But it is critical to highlight that calculated metrics don’t always correlate well with the wet-lab. Higher diversity doesn’t automatically translate to better therapeutics. These designs need to be tested for function and developability.</p><h3>Transparency: Variable</h3><p>They report the ubiquitin memorization bug (73–76 residue designs collapse to ubiquitin sequences because it appears &gt;1000x in PDB). They describe the v0 template bug that made filtering nearly useless. They honestly report weak small molecule affinities (50–150 µM for rucaparib) rather than overselling proof-of-concept results.</p><p>On the other hand, they didn’t perform a systematic failure analysis for why 3/9 novel targets failed. They didn’t report filter pass-through rates or perform binding site validation.</p><p>The penguinpox and hemagglutinin results are 1/7 and 7/7 nanobody hits in yeast display, with binding “at best 2 µM” based on assay limits. They honestly stated that the binding signals are weak.</p><h3>Open Source Release</h3><p>MIT license on model weights, training code, inference pipeline, and all designs. You can reproduce this, modify the architecture, and build on it.</p><p>* The computational work was led by the <a href="https://people.csail.mit.edu/tommi/">Tommi Jaakkola</a> and <a href="https://www.rbg.mit.edu/">Regina Barzilay</a> labs at MIT and experimental validation was performed by the <a href="https://pharm.ucsf.edu/degrado">William F. DeGrado lab</a> (UCSF) for bioactive peptides and small molecules, <a href="https://www.molgen.mpg.de/hnisz-lab">Denes Hnisz lab</a> (Max Planck Institute) for NPM1, <a href="https://www.davidsabatinilab.com/">David M. Sabatini lab</a> (IOCB) for RagC/RagA:RagC, <a href="https://liulab.com/ccl/">Chang C. Liu lab</a> (UC Irvine) for penguinpox and hemagglutinin, <a href="https://biology.mit.edu/profile/gene-wei-li/">Gene-Wei Li lab</a> (MIT) for GyrA antimicrobials, and <a href="https://www.adaptyvbio.com/">Adaptyv Bio</a> for the 9 novel + 5 benchmark protein targets.</p><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><h3>¹ Appendix: Understanding BoltzGen’s Ranking Metrics</h3><p>BoltzGen filters and ranks designs using several computational metrics. Here’s what each one measures and how it affects filtering:</p><h4>Confidence Metrics (Model’s Self-Assessment)</h4><p><strong>design_iiptm (interface predicted template modeling score)</strong></p><ul><li><strong>Measures:</strong> How confident the model is that the designed protein and target will form a stable binding interface</li><li><strong>Scale:</strong> 0 to 1, higher is better</li><li><strong>Role in filtering:</strong> A design might fold well on its own but fail to make good contacts with the target. This metric catches that problem.</li><li><strong>Limitation:</strong> High confidence doesn’t guarantee actual binding — the model can be confidently wrong.</li></ul><p><strong>design_ptm (predicted template modeling score)</strong></p><ul><li><strong>Measures:</strong> How confident the model is in the overall 3D structure of the designed protein</li><li><strong>Scale:</strong> 0 to 1, higher is better</li><li><strong>Role in filtering:</strong> If the model isn’t confident the design will fold into the predicted shape, it probably won’t work experimentally.</li><li><strong>Limitation:</strong> Measures structure confidence, not whether that structure will actually bind the target.</li></ul><p><strong>neg_min_design_to_target_pae (negative minimum predicted aligned error)</strong></p><ul><li><strong>Measures:</strong> The predicted positional uncertainty between designed protein and target atoms. PAE tells you “if I know where atom A is, how uncertain am I about where atom B is?”</li><li><strong>Scale:</strong> Negative values (lower PAE = higher negative value = better). They use the negative so higher numbers are better across all metrics.</li><li><strong>Role in filtering:</strong> Low PAE between binder and target means the model is confident about the exact binding geometry.</li><li><strong>Limitation:</strong> This is the model’s confidence, not experimental validation.</li></ul><h4>Physics-Based Metrics (Actual Molecular Interactions)</h4><p><strong>Hydrogen bonds</strong></p><ul><li><strong>Measures:</strong> Number of predicted hydrogen bonds between designed protein and target</li><li><strong>Role in filtering:</strong> Hydrogen bonds drive binding specificity and affinity. More isn’t always better (geometry matters), but too few suggests weak binding.</li><li><strong>How it’s calculated:</strong> Geometric criteria — typically donor-acceptor distance &lt;3.5 Å and favorable angles.</li></ul><p><strong>Salt bridges</strong></p><ul><li><strong>Measures:</strong> Number of predicted electrostatic interactions between oppositely charged residues (e.g., lysine-aspartate pairs)</li><li><strong>Role in filtering:</strong> Salt bridges contribute to binding energy and can provide specificity. But they’re pH-sensitive and can be destabilizing in hydrophobic environments.</li><li><strong>Limitation:</strong> Counting salt bridges doesn’t tell you if they’re actually stabilizing or if they’ll form in experimental conditions.</li></ul><p><strong>Buried surface area</strong></p><ul><li><strong>Measures:</strong> Amount of protein surface that becomes hidden from solvent when the designed protein binds the target</li><li><strong>Scale:</strong> Typically measured in Ų (square Angstroms)</li><li><strong>Role in filtering:</strong> More buried surface area generally means stronger binding (correlates with affinity). Also indicates how much of a conformational change might be needed for binding.</li><li><strong>Typical values:</strong> Antibody-antigen interfaces bury 600–900 Ų per chain.</li></ul><h4>The Worst-Rank-Across-Metrics Algorithm</h4><p>Instead of averaging scores or weighting them, BoltzGen ranks each design on every metric independently, then assigns the design its worst rank.</p><p><strong>Example:</strong></p><ul><li>Design A ranks: 5th (iiptm), 3rd (ptm), 500th (PAE), 10th (H-bonds), 8th (salt bridges), 12th (buried area)</li><li>Design A’s final rank: <strong>500th</strong> (its worst performance)</li></ul><p>This eliminates designs with any single critical flaw. A design that scores great on 5 metrics but terrible on 1 is probably broken in a way that matters experimentally. Forces designs to be well-rounded rather than excellent in some areas and terrible in others.</p><p>But it might reject designs that have one legitimately bad score but would still work. And it assumes all metrics are equally important — the worst rank in any category is disqualifying.</p><h4>Metric Weight Calibration</h4><p>BoltzGen calibrated these metric weights on 11,000 validated binders from previous work. They adjusted how much each metric contributes to filtering based on what actually worked in experiments.</p><p>But we don’t know which previous binders were used for calibration, whether those binders are similar to the new targets being designed, or how well calibration on known binders generalizes to novel targets.</p><h4>Filter Pass-Through Rates: Not Reported</h4><p>The paper doesn’t report how many of the 60,000 generated designs pass initial thresholds for each metric, how many designs survive after applying all filters, whether failed designs failed on one metric or many, or what the typical score distributions look like.</p><p>We can’t tell if the filters are highly selective (60,000 → 20 → test 15) or if most designs pass (60,000 → 10,000 → pick 15 randomly). That affects how we interpret the 66% experimental success rate.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef3229e22af4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[JAM-2: Performance Metrics and Validation Gaps]]></title>
            <link>https://medium.com/@enginyapici/jam-2-performance-metrics-and-validation-gaps-51f40a6511f2?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/51f40a6511f2</guid>
            <category><![CDATA[protein-engineering]]></category>
            <category><![CDATA[antibody]]></category>
            <category><![CDATA[biotechnology]]></category>
            <category><![CDATA[drug-discovery]]></category>
            <category><![CDATA[antibody-engineering]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 02 Dec 2025 05:59:34 GMT</pubDate>
            <atom:updated>2025-12-18T19:00:22.569Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>I Stress-Tested the Claims in Nabla’s JAM-2 Paper. These Are the Gaps.</strong></h3><p><em>Technical analysis of Nabla Bio’s computational antibody design platform</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sG7pboAja7EQ1MLet2dKCQ.jpeg" /><figcaption>JAM-2 designed antibodies binding to four different targets (PRL, NTM1A, AHSP, PARVA) with picomolar to low-nanomolar affinities. BLI sensorgrams show concentration-dependent binding. Computational models (left) predict binding poses, but no experimental structures validate these predictions.</figcaption></figure><p><a href="https://www.nabla.bio/news/denovo">Nabla Bio</a>’s <a href="https://nabla-public.s3.us-east-1.amazonaws.com/2025_Nabla_JAM2.pdf">JAM-2 paper</a> reports 100% target coverage, 39% success for VHH-Fcs and 18% for mAbs, and picomolar affinities on multiple targets.</p><p>The GPCR targeting is the strongest part with 11.7% success on CXCR4 <strong>directly on cells</strong>. The 57% developability pass rate: designs with good biophysical properties from the start. And the four-week timeline for 16 targets is impressive.</p><p>The paper also has some validation gaps. Critical controls are missing, the epitope targeting claim lacks experimental verification (<em>see Addendum 1 at the end</em>), and key methodological details aren’t disclosed. Some claims extend beyond what the experimental data demonstrates.</p><p>At a minimum, a binning experiment is needed for the epitope breadth claim (<em>see Addendum 1 at the end</em>). A proper control library needs to be added to the paper. GPCR binders should be tested for agonism or antagonism using a cell-based assay.</p><h3>What the Paper Reports</h3><h4><strong>Target coverage and affinities:</strong></h4><p>For Target Set 1, they selected 16 proteins that had no close homology to any antibody-antigen complex in the model’s training data. Even structurally similar proteins weren’t in the training set.</p><p>From 45 designs tested per format per target (1,440 total BLI readings at 500 nM antibody concentration):</p><ul><li>VHH-Fc: ~18 binders per target on average (39% rate)</li><li>Full-length mAbs: ~8 binders per target on average (18% rate)</li></ul><p>Success is defined as &gt;0.1 nm shift in BLI signal above background (empty supernatant reference). No affinity cutoff.</p><p>From these ~411 total binders across all targets, they selected up to 10 per format per target for affinity characterization. Measured affinities ranged widely:</p><ul><li>Picomolar binders on 2/16 targets</li><li>Single-digit nanomolar on 7/16 targets</li><li>Double-digit nanomolar on 11/16 targets</li></ul><p><strong>This means a 100+ nM binder counts as “success.”</strong> The actual high-quality binder rate (&lt;10 nM) is much lower than the reported success rates.</p><p><strong>Epitope targeting (Target Set 2):</strong></p><p>For 10 targets, they selected 20 epitopes each spanning different physicochemical properties. For each epitope, JAM-2 designed ~600 VHH sequences (12,000 designs per target total).</p><p>Workflow:</p><ol><li>Clone all designs into yeast display libraries (separated by epitope)</li><li>Screen against full-length antigen at multiple concentrations (1 μM, 100 nM, 10 nM, 1 nM)</li><li>Also screen against ovalbumin to filter polyspecific binders</li><li>Designs binding at 100 nM target without binding ovalbumin = success</li><li>Express subset as VHH-Fc in mammalian cells for affinity measurement</li></ol><p>Results by target:</p><ul><li>Best epitopes: ~30–90% success rates on yeast display</li><li>Epitope coverage: 30–70% of all specified epitopes yielded binders for 5/10 targets</li><li>All 10 targets yielded at least some epitope-specific binders</li><li>Confirmed single- to double-digit nanomolar affinities for selected hits</li></ul><p><strong>GPCR results:</strong></p><p>~730 designs each for CXCR4 and CXCR7, targeting orthosteric pockets. Tested on cells expressing the receptors.</p><ul><li>CXCR4: 11.7% on-cell bind rate (86/736 designs)</li><li>CXCR7: 3.8% on-cell bind rate (28/728 designs)</li><li>Binders defined as ≥3-fold signal over parental cell line</li><li>Top affinities: 1.4 nM (CXCR4), 23 nM (CXCR7)</li><li>Target-specific: no cross-reactivity between the two receptors</li></ul><p><strong>Developability profiling:</strong></p><p>923 antibodies characterized across:</p><ul><li>Expression titer (normalized to trastuzumab)</li><li>Monomericity by SEC</li><li>Hydrophobicity by PAIA-HIC</li><li>Polyspecificity by BVP ELISA</li></ul><p>Overall: 57% meeting all four criteria. Individual properties typically passed at 80%+.</p><h3>Now Let’s Poke Holes</h3><h4>Issue #1: The Epitope Claim is Unvalidated</h4><p>(<strong><em>This gap has been addressed. See Addendum 1 at the end.</em></strong>)</p><p>For anyone who has actually run epitope validation experiments, this is the biggest problem in the paper.</p><p><strong>What they claim:</strong></p><ul><li>“JAM-2 achieves state-of-the-art epitope breadth, successfully targeting user-specified epitopes”</li><li>“30–70% of epitopes for half the targets”</li><li>“This transforms antibody discovery from a stochastic process into something approaching rational design”</li></ul><p><strong>What they actually did:</strong></p><p>For each of 20 epitopes per target, they designed 600 VHHs and tested them all against the <strong>same full-length antigen</strong>. They didn’t use epitope-specific reagents, utilize domain deletions or peptides. They just used full-length proteins.</p><p>A binder designed for “epitope 1” and a binder designed for “epitope 20” get the same experimental readout: binds full-length antigen at 100 nM, doesn’t bind ovalbumin. Both called “successes” for their respective epitopes.</p><p><strong>They didn’t perform epitope validation:</strong></p><ul><li>No competitive binding assays</li><li>No epitope binning</li><li>No blocking experiments (even though they already had hundreds of binders in hand)</li><li>No structural validation of any designed complex</li><li>Nothing to confirm binders designed for different epitopes bind different sites</li></ul><p>In principle, all designs per target could be binding the same immunodominant patch, and their assay wouldn’t detect it.</p><p>They show computational models of designs bound to different epitopes (Figure 2e), implying experimental confirmation. But these are just predictions.</p><p>If I were reviewing this for a journal, this is exactly the point where I would ask for binning data before letting the epitope claim stay.</p><p><strong>Why additional validation would strengthen these claims:</strong></p><p>The framing implies validated epitope specificity. A reasonable reader (especially someone used to seeing binning data tucked into a supplemental figure) would assume “epitope-specific binder” means they actually confirmed epitope specificity.</p><p>With hundreds of binders per target, basic epitope binning would be straightforward to add and would substantially strengthen these claims.</p><p><strong>The honest reporting:</strong> “We designed VHH libraries targeting 20 different epitopes per target. X% of epitope-targeted libraries yielded binders to the full-length antigen. Computational models predict these binders engage their specified epitopes, but experimental epitope mapping was not performed.”</p><p><strong>What they wrote instead:</strong> Framed as demonstrated epitope breadth, put it in figure titles, compared favorably to traditional methods.</p><h4>Issue #2: A Critical Omission: No Framework Information</h4><p>The paper doesn’t report what framework(s) they use. (<strong><em>This gap has been addressed. See Addendum 2 at the end.</em></strong>)</p><p>They tell us JAM-2 designs “the full set of CDRs (IMGT definition), along with a small number of adjacent framework 2 and 3 residues that contact the antigen.”</p><p>But not what scaffold these CDRs are grafted onto. Which human germline? Single framework or multiple? How are the non-designed framework regions (~70% of the variable domain) selected?</p><p><strong>The paper is otherwise extremely detailed:</strong></p><ul><li>Exact plasmids (pcDNA3.4)</li><li>Expression conditions</li><li>Biosensor regeneration buffers</li><li>Every developability assay protocol</li></ul><p>But nothing about antibody frameworks.</p><p><strong>Why this is important:</strong></p><ul><li>Framework choice significantly affects expression, stability, and binding</li><li>We can’t reproduce the work</li><li>We can’t design proper controls</li><li>We don’t know if success depends on their framework choice</li></ul><p>For a paper that says it can support front-line drug discovery, leaving out the framework is the equivalent of publishing a small-molecule hit without showing the scaffold.</p><h4><strong>Issue #3: No Disclosure of Structural Inputs to the Model</strong></h4><p>The paper doesn’t specify what structural information JAM-2 uses as input. Do they feed in crystal structures? AlphaFold predictions? Domain-level models? Or just raw sequences?</p><p>For closed, proprietary models, reproducibility isn’t the expectation because no one outside the company can rerun the model anyway. But transparency about the model’s inputs is still essential because it determines how to interpret the results. Without knowing whether JAM-2 relies on structural templates, predicted models, or purely sequence-based featurization, it’s impossible to assess:</p><ul><li>how much of the performance is driven by structural context,</li><li>how sensitive the model is to errors in predicted structures,</li><li>whether the approach generalizes across targets with limited structural data,</li><li>or how to evaluate the claimed epitope targeting.</li></ul><h4>Issue #4: No Negative Control Library</h4><p>They tested designs against empty ExpiCHO supernatant, but <strong>never tested a control library of random sequences.</strong></p><p>What would a proper control look like?</p><ul><li>Random CDR sequences on the same framework(s) JAM-2 uses</li><li>Tested identically with the same &gt;0.1 nm threshold</li></ul><p>This would answer: what’s the baseline hit rate for non-designed sequences?</p><p>Without this, we don’t know if 39% is impressive (vs. 5% for random) or barely above noise (vs. 30% for random).</p><p>We can’t even design this control ourselves because they don’t report the frameworks.</p><p>Chai-2 validated their designs with five cryo-EM structures that match predictions within &lt;2 Å. With that level of structural proof, negative controls become less critical.</p><p>JAM-2 has no experimental structures. Without that validation, you need a proper negative-control library to show the success rates reflect design capability rather than permissive assays. In short: <strong>either show structures or add a control library.</strong></p><h4>Issue #5: No Experimental Structures, Despite Claiming Structural Precision</h4><p>For a computational design paper that claims structural precision or epitope specificity, the lack of cryo-EM or crystal structures is a major gap.</p><p><strong>Chai-2 now has 5 solved structures showing &lt;2Å agreement with predictions.</strong> JAM-2 has computational models only.</p><p>Without experimental structures, we don’t know if designs actually adopt the predicted binding modes. And computational models routinely miss important details.</p><p><strong>JAM-2 doesn’t explicitly claim “structural accuracy” but they:</strong></p><ul><li>Show structural models throughout the paper</li><li>Discuss “geometry of recognition” and “intentionally designing the geometry of engagement”</li><li>Present models as if they represent reality</li></ul><p>This creates an implicit claim of structural precision. But they provide no experimental validation.</p><p><strong>This gap is particularly critical for the epitope claims.</strong> Experimental structures would directly prove or disprove whether binders designed for different epitopes actually bind different sites. They have hundreds of binders. Solving even 5–10 structures would validate this core claim.</p><p><strong>Most of these gaps are addressable. Here’s what Nabla can do.</strong></p><h3>What Needs to Be Done</h3><p>Many of the gaps identified above can be addressed through straightforward additions to the paper, either by disclosing existing information or by running targeted validation experiments.</p><p><strong>Can be addressed by updating the paper with existing information:</strong></p><ul><li>Disclose the framework(s) used for VHH and mAb designs</li><li>Specify what structural information JAM-2 uses as input (crystal structures, AlphaFold predictions, or sequences)</li></ul><p><strong>Can be addressed through additional validation experiments:</strong></p><ul><li>Epitope binning or competition assays for a representative set of designs from Target Set 2 (straightforward with hundreds of binders in hand).</li><li>Test at least one negative control library: random CDR sequences on the same framework(s), tested under identical conditions. Alternatively: solve some structures.</li><li>Functional characterization of GPCR binders (straightforward with commercial kits).</li><li>Bonus: Solve experimental structures (cryo-EM or X-ray) for a subset of designed complexes to validate predicted binding modes.</li></ul><p>The epitope mapping experiments would be particularly valuable and relatively simple to execute. With hundreds of binders already expressed and validated, running competition assays or binning experiments on even 20–30 representative designs would directly test whether different epitope-targeted libraries actually bind different sites.</p><p>Similarly, testing a single negative control library (random CDRs on the disclosed framework) would provide essential context for interpreting the 39% success rate.</p><p>For a company positioning this as ready for industrial use, these are fundamental gaps that prevent independent validation of the core claims. For partners evaluating the platform, the epitope specificity claim needs experimental binning. Framework disclosure and negative controls would provide the validation diligence typically requires.</p><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><h4>Addendum 1: Epitope Claim Validation (Updated December 18, 2025)</h4><p><strong>Original concern in my post:</strong></p><blockquote><em>“They performed zero epitope validation: No competitive binding assays, epitope binning, or blocking experiments.”</em></blockquote><p><strong>Update from </strong><a href="https://informaconnect.com/antibody-engineering-therapeutics/speakers/surge-biswas-phd/#track-1-emerging-in-vitro-approaches-to-antibody-discovery-livestream_jam-2-fully-computational-design-of-drug-like-antibodies-with-high-success-rates"><strong>Surge Biswas’s presentation</strong></a><strong> at </strong><a href="https://informaconnect.com/antibody-engineering-therapeutics/"><strong>AET</strong></a><strong> on December 17, 2025:</strong></p><p>Nabla has performed epitope binning experiments in another program and Surge presented data for 2 targets during the talk. This directly addresses the validation gap identified in Issue #1.</p><p>The epitope binning data, if it shows designs from different epitope-targeted libraries binding to distinct sites, would substantiate the epitope breadth claim. This is the type of validation needed and was the minimum requirement I outlined in the post.</p><p>I look forward to seeing this data published in the peer-reviewed version or supplementary materials with the targets in JAM-2 report. Epitope binning plus even a few X-ray or cryo-EM structures confirming predicted binding modes would fully address this concern.</p><h4>Addendum 2: Framework Information Disclosed (Updated December 18, 2025)</h4><p><strong>Original concern in my post:</strong></p><blockquote><em>“The paper doesn’t report what framework(s) they use.”</em></blockquote><p><strong>Update from </strong><a href="https://informaconnect.com/antibody-engineering-therapeutics/speakers/surge-biswas-phd/#track-1-emerging-in-vitro-approaches-to-antibody-discovery-livestream_jam-2-fully-computational-design-of-drug-like-antibodies-with-high-success-rates"><strong>Surge Biswas’s presentation</strong></a><strong> at </strong><a href="https://informaconnect.com/antibody-engineering-therapeutics/"><strong>AET</strong></a><strong> on December 17, 2025:</strong></p><p>JAM-2 uses Specifica molecules across 5 different frameworks. This provides good structural coverage and addresses the reproducibility concern raised in Issue #2.</p><p>With framework information disclosed, the work becomes reproducible. The use of 5 different frameworks (rather than a single framework) also suggests the success rates aren’t artifacts of a particularly permissive scaffold choice.</p><p>This information should be included in the published version of the paper.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=51f40a6511f2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Tried to Poke Holes in Chai-2’s Antibody Design Paper. Here’s What I Found.]]></title>
            <link>https://medium.com/@enginyapici/i-tried-to-poke-holes-in-chai-2s-antibody-design-paper-here-s-what-i-found-7e51f5581c7d?source=rss-a009c1fc1b90------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e51f5581c7d</guid>
            <category><![CDATA[antibody]]></category>
            <category><![CDATA[therapeutics]]></category>
            <category><![CDATA[biotechnology]]></category>
            <category><![CDATA[drug-discovery]]></category>
            <category><![CDATA[protein-engineering]]></category>
            <dc:creator><![CDATA[Engin Yapici]]></dc:creator>
            <pubDate>Tue, 25 Nov 2025 04:58:42 GMT</pubDate>
            <atom:updated>2025-12-01T00:39:56.699Z</atom:updated>
            <content:encoded><![CDATA[<p><em>88 designed antibodies, atomic accuracy, and some important caveats.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EjwQptSSpnUGOMT2CFMdpw.png" /><figcaption>Designed structure (gray) vs. experimental reality (blue): RMSDs ranging from 0.17Å to 1.70Å. Compare this to RFdiffusion’s one published structure, which showed the wrong binding orientation entirely.</figcaption></figure><p>The <a href="https://www.chaidiscovery.com/news/chai-2-mab">Chai Discovery</a> team just released <a href="https://chaiassets.com/chai-2/paper/technical_report_challenging_targets.pdf">a technical report</a> on designing therapeutic antibodies with their Chai-2 platform. This appears to be the most advanced computational antibody design capability demonstrated publicly.</p><p>I’m going to walk through what they actually achieved, then try my hardest to find every weakness, shortcut, and oversold claim I can. Turns out: they accomplished some genuinely impressive firsts, and they were remarkably transparent about the limitations.</p><h3>What They Actually Achieved</h3><h4>First Computational Design of Full-Length mAbs at Scale</h4><p>88 full-length IgG1 monoclonal antibodies designed and validated, with 93% of scFv hits retaining activity when reformatted. This is unprecedented.</p><p>Every other computational approach (<a href="https://www.bakerlab.org/2023/07/11/diffusion-model-for-protein-design/">RFdiffusion</a>, <a href="https://boltz.bio/boltzgen">BoltzGen</a>, etc.) has focused on single-domain antibodies: VHHs or scFvs. Designing functional full-length IgGs means solving:</p><ul><li>VH-VL interface stability</li><li>Pairing compatibility</li><li>Maintaining activity in the therapeutic format</li><li>Dealing with Fc-mediated complications</li></ul><p>Nobody else has published anything close to this for mAbs. This alone is a significant first.</p><h4>Atomic-Level Structural Accuracy</h4><p>Five antibody-antigen complexes solved by cryo-EM, all matching computational predictions with RMSDs from 0.41 to 1.7 Å. This is atomic-level precision across:</p><ul><li>Different target classes (soluble proteins, membrane proteins)</li><li>Different antibody formats (Fabs)</li><li>Different epitope types</li></ul><p>Compare to RFdiffusion: <a href="https://www.nature.com/articles/s41586-025-09721-5">their one published cryo-EM structure</a> showed the antibody binding in a completely different orientation than predicted. When your structural model is wrong, you can’t rationally engineer the molecule.</p><p>Chai-2’s structural predictions are trustworthy enough to use for optimization.</p><h4>Therapeutic-Grade Developability Without Optimization</h4><p>Designs that expressed and purified showed:</p><ul><li>100% pass rate on thermal stability (all &gt;64°C)</li><li>79–90% pass rates on polyreactivity, hydrophobicity, and self-association</li><li>86% with three or more “green flags” across the developability panel</li><li>Thermostability distribution exceeding clinical antibody controls</li></ul><p>Traditional antibody discovery: screen millions, find binders, then spend 6–12 months fixing stability, aggregation, and expression issues. Many promising binders fail during developability optimization.</p><p>Chai-2 designs start with therapeutic-grade biophysics on the first iteration.</p><h4>GPCR Binders and Agonists</h4><p>Six GPCRs targeted, 100% produced at least one binder. Two produced functional agonists (CCR8, CXCR4): the first computationally designed GPCR agonist antibodies demonstrated.</p><p>Traditional antibody discovery essentially fails for GPCR agonists. Display libraries can’t handle membrane proteins well. The therapeutic landscape has three GPCR-targeting antibodies total; none are agonists.</p><p>Getting measurable agonism from computationally designed antibodies opens a design space that was previously inaccessible.</p><h4>Single-Residue Selectivity on pMHC</h4><p>1.5 nM affinity to HLA-A*03:01-KRAS G12V from 50 designs, with selectivity against:</p><ul><li>G12D (single amino acid difference)</li><li>Wildtype KRAS (single amino acid difference)</li><li>Same peptide on HLA-A*11:01 (different MHC allele)</li></ul><p>Designing any antibody with nanomolar affinity to a pMHC complex is difficult. The epitope is 2–4 exposed peptide residues on a relatively flat surface. Traditional methods screen 10⁹+ variants via display, get hit rates below 0.001%, then require extensive affinity maturation.</p><p>Chai-2: 4% hit rate from rational design, nanomolar affinity, qualitative single-residue discrimination.</p><h4>Transparency in Reporting</h4><p>They clearly disclosed:</p><ul><li>Framework selection (VH3–23 and VH3–66 chosen for high thermostability)</li><li>Exact exclusion numbers (7/88 IgGs, 10/27 VHHs failed expression)</li><li>All PDB codes for structures used</li><li>Variable success rates by target in detailed tables</li><li>Direct acknowledgment this was proof-of-concept, not clinical candidates</li><li>Statement that designs “were designed to validate the model rather than to meet a patient-oriented target product profile”</li></ul><p>Most papers bury failures or report only best cases. The criticisms I’ll detail below come from reading their openly reported methods, not detective work.</p><h3>Now Let’s Poke Holes</h3><h4>Issue #1: The GPCR Agonist Success Relied on Near-Perfect Structural Information</h4><p>For CCR8 (their star agonist example), they used PDB structure 8KFX, which shows:</p><ul><li>The receptor in its active conformation</li><li>The small molecule agonist LMD-009 bound to the orthosteric pocket</li><li>The G-protein stabilizing the active state</li></ul><p>This is essentially a blueprint showing exactly how to activate this receptor. Their designed antibody places “CDRH3 F105 in this same region” where LMD-009 binds.</p><p>For their other GPCR targets:</p><ul><li>CCR2: PDB 7XA3</li><li>CCR5: PDB 6AKX</li><li>CXCR4: PDB 8U4R</li><li>CXCR6: Structure available but not specified</li></ul><p><strong>The scalability problem:</strong> There are only a handful of GPCRs with high-resolution active-state structures showing bound agonists. The PDB has ~800 GPCR structures total, but only a tiny fraction show active states with co-crystallized ligands in the orthosteric pocket.</p><p>CCR8 structure 8KFX was published in 2024. It’s one of very few cases where all the necessary structural information exists to template an agonist design.</p><p>The paper frames this as “designing GPCR agonists,” implying a generalizable capability. What they actually demonstrated: “if you have an active-state structure with a known agonist, we can design an antibody that mimics it.”</p><p>That said: nobody else has done even this much. Traditional methods fail on membrane proteins. Other computational approaches haven’t produced functional GPCR binders, let alone agonists. Chai-2 took structural information (the smart thing to use) and converted it into functional designs from small pools (10 candidates tested for CCR8). Execution matters.</p><h4>Issue #2: Success Rates Vary Wildly Depending on Target</h4><p>GPCR results from Table 2:</p><ul><li>CCR5: 13/13 designs bound (100%)</li><li>GPRC5D: 35/73 bound (48%)</li><li>CCR2: 10/40 VHH bound (25%)</li><li>CCR8: 5/10 bound (50%), but only 1 showed agonism (10%)</li><li>CXCR4: 2/19 bound (11%), 1 partial agonist (5%)</li><li>CXCR6: 1/23 bound (4%), 0 functional</li></ul><p>pMHC results:</p><ul><li>KRAS G12V: 2/50 designs bound (4%)</li><li>p53-R175H: 0/27 (0%)</li><li>gp100: 0/48 (0%)</li></ul><p>The paper emphasizes “100% target success rate”, meaning at least one binder to each of six GPCRs. But per-design success swings from 4% to 100%.</p><p>Compare this to traditional pMHC discovery: screening 10⁹+ variants via display, hit rates below 0.001%, extensive affinity maturation required. Chai-2’s 4% from 50 rational designs is orders of magnitude more efficient.</p><p>RFdiffusion/RFAntibody reported 10–20% hit rates for VHHs against standard protein targets. Chai-2’s rates on challenging targets (GPCRs, pMHCs) range from 4% to 50%, with one outlier at 100%.</p><p>The variability just means this isn’t a solved problem; it’s target-dependent, and some targets are genuinely hard.</p><h4>Issue #3: Framework Selection Biased the Thermostability Results</h4><p>The paper emphasizes that all Chai IgG designs exceeded 64°C thermal stability, with a distribution “markedly exceeding” biosimilar controls.</p><p>From the discussion: “This trend reflects our deliberate selection of the well-behaved VH3–23 and VH3–66 frameworks, which are associated with high thermostability in the Jain et al. [8] dataset.”</p><p>They clearly disclosed this. But it means the thermostability results are substantially attributable to framework choice, not purely the model’s ability to generate stable sequences. These frameworks were specifically selected from a dataset for their high thermostability; that’s literally why they were chosen.</p><p>We don’t know what would happen with a broader range of frameworks. Not every framework will be suitable for a given target. Structural geometry constrains which epitopes can be effectively engaged.¹ A robust design model should work across multiple frameworks with different packing angles (i.e., with different VH-VL orientations). The variable success rates (4% to 100% across targets) might partly reflect which epitopes happen to be geometrically compatible with VH3–23/VH3–66.</p><p>Choosing stable frameworks is smart rational design. The achievement isn’t the Tm numbers themselves; it’s the full-length mAb design capability.</p><h4>Issue #4: Survivor Bias in Developability Statistics</h4><p>The paper reports “&gt;86% of designs have three or more developability green flags.”</p><p>From the methods: “This criterion led to the exclusion of 1/20 IgG biosimilar controls, 7/88 Chai IgGs, 3/20 VHH-Fc controls, and 10/27 Chai VHH-Fcs.”</p><p>They reported exact numbers and that’s transparent. But they excluded:</p><ul><li>7/88 IgGs (8%)</li><li>10/27 VHHs (37%)</li></ul><p>These didn’t purify well enough for biophysical characterization. Developability statistics are calculated on what remained.</p><p>If you count “failed to express at &gt;90% purity” as a developability failure:</p><ul><li>IgGs: 7/88 failed expression, plus some fraction of the remaining 81 failed other assays</li><li>VHHs: 10/27 failed expression (37%)</li></ul><p>Still, compare to traditional campaigns: 30–50% of hits routinely fail during developability assessment. Many require extensive engineering to fix aggregation or stability.</p><p>RFdiffusion/RFAntibody published no developability data. Other generative models often produce sequences that don’t express at all.</p><p>Chai-2’s designs that DO express show thermal stabilities exceeding many approved therapeutics. The 37% VHH expression failure rate is concerning but still better than many computational approaches producing entirely non-functional sequences.</p><h4>Issue #5: Incomplete Selectivity Data for pMHC</h4><p>The KRAS G12V result: 1.5 nM affinity, discriminates against G12D, wildtype, and mismatched MHC allele.</p><p>Figure 5C shows ELISA curves. At 10 µg/mL, design_46 binds the target strongly and shows “negligible binding” to off-targets.</p><p>But they never measured actual KD values for the off-targets. What’s the KD against G12D? Against wildtype? Against mismatched MHC?</p><p>“Negligible binding at 10 µg/mL” doesn’t quantify selectivity. Is it 100-fold? 1000-fold? 10,000-fold? Without kinetics on off-targets, we can’t assess true single-residue discrimination vs. just picking a concentration where one binder works and others don’t.</p><p>The qualitative selectivity (binds target, doesn’t bind closely related variants at the same concentration) is consistent with genuine specificity. They should have done the kinetics.</p><p>The on-target performance (nanomolar affinity from 50 designs to a difficult epitope) remains impressive. RFdiffusion got micromolar affinity using yeast display with millions of combinatorial variants.</p><h4>Issue #6: Cherry-Picked Cryo-EM Structural Validation</h4><p><em>Update: See addendum at the end of section.</em></p><p>Five cryo-EM structures, all showing excellent agreement (RMSDs 0.41–1.7 Å). Genuinely remarkable accuracy.</p><p>But: “five selected designs.”</p><p>Selected from what pool? Did they attempt structural studies on other designs that showed poor agreement? The supplement analyzes the five that worked but says nothing about whether others were attempted or how these five were chosen.</p><p>If they attempted 10 structures and five matched well, that’s good but not exceptional. If they attempted five and all five matched, that’s extraordinary. We don’t know.</p><p>Even if cherry-picked, demonstrating sub-2 Å RMSD across five different antibody-antigen pairs, different target classes, different formats: this is far beyond what anyone else has shown. RFdiffusion: 1 structure, wrong binding mode. When Chai-2’s structural predictions are accurate, you can use them for structure-guided optimization and rational engineering. When your model predicts the wrong binding orientation, you can’t.</p><h4><em>Addendum: Cryo-EM Structure Selection</em></h4><p><strong>Clarification from Chai Discovery: </strong><a href="https://www.linkedin.com/in/joshua-meier-27a6861a/">Joshua Meier</a> (Chai-2 co-author) <a href="https://www.linkedin.com/feed/update/urn:li:activity:7399103326447026176?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7399103326447026176%2C7399200337573867520%29&amp;dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287399200337573867520%2Curn%3Ali%3Aactivity%3A7399103326447026176%29">clarified</a> that all five cryo-EM structures attempted matched predictions (5/5). Designs were selected upfront to represent diverse targets and binding modes, not filtered post-hoc.</p><p><strong>Updated assessment:</strong> 5/5 success rate across diverse targets, formats, and binding modes is exceptional and strengthens the structural accuracy claim.</p><h4>Issue #7: “First GPCR Agonist” Claims Are Oversimplified</h4><p>CCR8_design_06: EC50 of 272 nM vs. 40 nM for natural ligand CCL1 (7-fold weaker)</p><p>CXCR4_design_06: Partial agonist, EC50 ~164 nM vs. 0.83 nM for CXCL12 (200-fold weaker, incomplete activation)</p><p>Both were designed targeting orthosteric pockets using structures showing exactly where known agonists bind.</p><p>These aren’t optimized therapeutic agonists. They’re proof-of-concept molecules showing computationally designed antibodies can produce measurable receptor activation.</p><p>Nobody has designed GPCR agonist antibodies computationally before. Weaker than natural ligands just means room for optimization. Getting ANY agonism opens a previously inaccessible design space.</p><h4>Issue #8: Post-Hoc Rationalization</h4><p>For CCR8_design_06, the paper suggests it works because it has the “slowest dissociation rate” with “prolonged residence time.”</p><p>They didn’t design for slow off-rates. They designed for orthosteric engagement, tested what came out, one worked better, then measured kinetics and found slow dissociation.</p><p>They found what worked, then figured out why; not the other way around.</p><p>This is how drug discovery actually works. You design for a mechanism, test candidates, one performs better, you figure out why. The understanding feeds back into the next cycle.</p><p>The alternative would be claiming they predicted dissociation kinetics computationally and designed specifically for slow off-rates; even harder to believe.</p><p>Still: they designed 10 molecules targeting an orthosteric pocket, one produced genuine agonism, and they can analyze why for future iterations. That’s rational drug discovery.</p><h3>How this changes things in the field</h3><h4><strong>Newly tractable targets:</strong></h4><ul><li>GPCR agonists (huge unmet need, minimal therapeutic precedent)</li><li>Neoepitope-specific antibodies (single-residue discrimination for personalized cancer)</li><li>Conformationally selective binders (active vs inactive states)</li><li>Minimal surface area epitopes</li></ul><h4><strong>Compressed timelines:</strong></h4><ul><li>Target to validated hit: weeks vs. months</li><li>Hit to lead: potentially skipped if developability already acceptable</li><li>Structural characterization: predicted before synthesis</li></ul><h4><strong>Economic shift:</strong></h4><ul><li>Traditional: $2–5M per campaign (screening dominates)</li><li>Computational: $50–200K (design + focused validation)</li></ul><h4><strong>Risk reduction:</strong></h4><ul><li>Know binding mode before making the molecule</li><li>Predict developability issues computationally</li><li>Test 10–100 candidates instead of millions</li></ul><h3>TL;DR</h3><h4>The limitations:</h4><ul><li>Template-based design on rare structures with perfect information</li><li>Variable success rates (4% to 100%)</li><li>Cherry-picked frameworks for high stability</li><li>Survivor bias in statistics</li><li>Incomplete selectivity measurements</li><li>Possibly cherry-picked structural validation</li><li>Oversimplified “first” claims</li><li>Post-hoc rationalization</li></ul><p>This isn’t a magic wand that works on every target. Success requires high-quality structural information. Some targets failed completely. The approach is target-dependent with significant variability.</p><h4>The achievements:</h4><ul><li>First AI-designed full-length mAbs at scale (88 functional IgGs)</li><li>Atomic-level structural accuracy (five complexes &lt;2 Å RMSD)</li><li>Therapeutic-grade developability on first iteration</li><li>First computationally designed GPCR agonists</li><li>Nanomolar affinity with single-residue selectivity from small design pools</li><li>Remarkable transparency in reporting</li></ul><p>The structural accuracy alone is beyond what anyone else has shown. The GPCR agonist work, even with its shortcuts, demonstrated something new. The full-length IgG designs with therapeutic properties compress timelines in ways traditional methods can’t match.</p><p>If this is what they’re willing to publish (including all the limitations), it suggests the underlying capability is robust enough to be useful in practice.</p><p><strong>The field just moved from “can we generate sequences that bind?” to “can we generate therapeutic-quality molecules with predictable binding modes and functional properties?”</strong></p><p><strong>Chai-2’s answer: sometimes, depending on the target, if you have good structural information.</strong></p><p>That “sometimes” is still far better than anyone else’s “rarely” or “not yet demonstrated.”</p><p>Too many acronyms in one post? <a href="https://docs.google.com/spreadsheets/d/19gFrMO86cmJZNHapsRdt3oXYhbznZOhe801kurqlMUY/edit?usp=sharing"><strong>Check out my biotech abbreviation cheat sheet</strong></a> and feel free to suggest additions.</p><p>Follow me on <a href="https://substack.com/@enginyapici"><strong>Substack</strong></a><strong>, </strong><a href="https://medium.com/@enginyapici"><strong>Medium</strong></a><strong>, </strong><a href="https://bsky.app/profile/enginyapici.bsky.social"><strong>Bluesky</strong></a><strong>, </strong>and<strong> </strong><a href="https://www.linkedin.com/in/enginyapici/"><strong>LinkedIn</strong></a><strong> </strong>for more posts on drug discovery, assay development, and screening workflows.</p><h3>Appendix</h3><h4>¹ Why Framework Choice Matters for Antibody Design</h4><p>When Chai-2 disclosed they “deliberately selected VH3–23 and VH3–66 frameworks associated with high thermostability,” this wasn’t just about cherry-picking for good developability stats. Framework choice has fundamental implications for what the AI can actually design.</p><h4>The Structural Constraint</h4><p>Antibody frameworks aren’t just passive scaffolds. They dictate the geometric constraints for CDR loop conformations. Think of the framework as the pillars of a bridge that determine where the cables (CDRs) must attach and what angles they can adopt.</p><p>This relationship was characterized by Chothia and colleagues in the 1980s-90s, who discovered that CDRs L1, L2, L3, H1, and H2 adopt a limited set of “canonical” conformations determined by:</p><ul><li>Loop length</li><li>Key framework residues at specific positions</li><li>The framework-framework interface geometry</li></ul><p>CDR H3 is the major exception. It’s the most variable and the primary determinant of binding specificity.</p><h4>Implications for Antibody Design</h4><p>Different frameworks position CDR loops at different angles and orientations. The framework choice determines:</p><ol><li><strong>Geometric constraints</strong>: The “approach angle” of your designed binding site</li><li><strong>VH-VL packing</strong>: How heavy and light chains come together, affecting all six CDRs</li><li><strong>CDR conformation preferences</strong>: Certain frameworks favor certain canonical loop structures</li></ol><p>When you fix the framework, you’re essentially saying: “We’ll use this one arm configuration for every target and just change the fingers (CDRs).”</p><h4>The Design Space Trade-off</h4><p>True <em>de novo</em> design would co-optimize framework and CDRs together, or at a minimum test multiple framework families. By constraining to VH3–23/VH3–66, Chai-2:</p><p><strong>Gained:</strong></p><ul><li>Reproducible success (these frameworks are well-characterized)</li><li>High developability (these frameworks are thermostable)</li><li>Simpler design problem (fewer degrees of freedom)</li></ul><p><strong>Lost:</strong></p><ul><li>Targets that need different framework geometries</li><li>Potentially better binding modes with alternative scaffolds</li><li>Full exploration of antibody design space</li></ul><p>This is a pragmatic engineering choice, not a scientific limitation. But it does mean success rates might improve significantly if the method could explore framework diversity. The 4% hit rate on some targets might jump to 20% with the right framework geometry.</p><p>The framework-CDR relationship isn’t completely rigid. There’s some conformational flexibility and post-binding induced fit. But the framework sets strong preferences for loop trajectories, CDR-CDR distances, and overall binding site architecture.</p><p><strong>In short:</strong> Framework choice is a fundamental constraint on what antibody geometries the AI can explore, not just an optimization parameter. This is one reason why the variable success rates (4–100%) might correlate with how well each target’s epitope geometry matches the fixed framework’s geometric preferences.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e51f5581c7d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>