<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Julfy Patlatus on Medium]]></title>
        <description><![CDATA[Stories by Julfy Patlatus on Medium]]></description>
        <link>https://medium.com/@julfy?source=rss-2b9cd0ab5025------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Julfy Patlatus on Medium</title>
            <link>https://medium.com/@julfy?source=rss-2b9cd0ab5025------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 06:59:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@julfy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[External Client Apps Are Not a Drop-In Replacement for Connected Apps — At Least Not for Package…]]></title>
            <link>https://medium.com/@julfy/external-client-apps-are-not-a-drop-in-replacement-for-connected-apps-at-least-not-for-package-c67e1f234520?source=rss-2b9cd0ab5025------2</link>
            <guid isPermaLink="false">https://medium.com/p/c67e1f234520</guid>
            <category><![CDATA[salesforce]]></category>
            <category><![CDATA[external-client-app]]></category>
            <category><![CDATA[independentsoftwarevendor]]></category>
            <category><![CDATA[managed-package]]></category>
            <category><![CDATA[connected-apps]]></category>
            <dc:creator><![CDATA[Julfy Patlatus]]></dc:creator>
            <pubDate>Fri, 17 Jul 2026 09:33:57 GMT</pubDate>
            <atom:updated>2026-07-17T09:35:02.785Z</atom:updated>
            <content:encoded><![CDATA[<h3>External Client Apps Are Not a Drop-In Replacement for Connected Apps — At Least Not for Package Development</h3><p>Salesforce is steadily deprecating Connected Apps in favor of External Client Apps (ECA). On paper it’s a sensible consolidation. In practice, if your team builds a <strong>managed package</strong> that ships an OAuth-based integration, ECA in its current form makes that development workflow substantially harder, and in some cases impossible to do cleanly. This post documents exactly why, with the errors and support responses to back it up.</p><h4>The core difference that breaks everything</h4><p>With a Connected App, the metadata didn’t need to physically exist in every org where developers worked on the package. An External Client App must be <strong>physically present</strong> — installed or deployed — in every org you’re developing in. Already a heavier footprint for a multi-developer team.</p><h4>Problem 1: Consumer Key/Secret are silently regenerated on deploy</h4><p>The ExtlClntAppGlobalOauthSettings metadata type exposes consumerKey and consumerSecret fields. Set them to fixed values. Deploy:</p><ul><li>Deploy <strong>succeeds</strong>.</li><li>Salesforce <strong>ignores</strong> the values you set.</li><li>Salesforce <strong>generates a new pair</strong> for that org.</li></ul><p>If the third-party system on the other end only accepts one static client ID/secret — the normal case — every deploy touching this metadata silently breaks the connection. No error, no warning, nothing in the deploy log.</p><p><strong>This is a real regression, not just “cross-org key reuse is unsupported.”</strong> Legacy Connected Apps handle the exact same scenario — deploying with keys that already exist elsewhere — by failing loudly. Two concrete cases:</p><p>If the Connected App is still deployed/present in the org:</p><pre>=== Deploy Errors (1) ===<br>ERROR: src/unpackaged/eca/connectedApps/App.connectedApp-meta.xml: You cannot provide a Consumer Key</pre><p>If the Connected App was deleted (key no longer attached to a live app, but still reserved):</p><pre>Starting metadata deployment...<br>=== Deploy Errors (1) ===<br>ERROR: src/unpackaged/eca/connectedApps/App1.connectedApp-meta.xml: The consumer key is already taken</pre><p>Both are explicit deploy failures. You know immediately something’s wrong and can react before anything breaks in production. ECA’s behavior — silently substituting credentials and reporting a clean deploy — removes that safety net entirely. The failure mode moved from “deploy-time error” to “integration silently stops authenticating, discovered later.”</p><p>I opened a case with Salesforce Support about this. Their response, verbatim (lightly trimmed):</p><blockquote><strong><em>1. Using the same Consumer Key and Secret across multiple orgs:</em></strong><em> Currently, there isn’t a supported way to deploy an External Client App with the same Consumer Key and Consumer Secret across multiple Developer Edition orgs or Scratch Orgs. The recommended approach is to create a Packaged External Client App in a persistent org (such as a Dev Hub or packaging org).</em></blockquote><ul><li>Sandboxes refreshed from the source org will use the same Consumer Key and Consumer Secret.</li><li>Scratch Orgs should reference the packaged External Client App using orgScopedExternalApp instead of creating a new one, e.g. Packaged / YourECALabel / [SOURCE-ORG-ID]:YourECALabel.</li><li>Developer Edition orgs do not support this approach, so if consistent credentials are required, Scratch Orgs or Sandboxes are the recommended environments.</li></ul><blockquote><strong><em>2. Why isn’t an error returned when consumerKey is specified?</em></strong><em> I understand why this is confusing. Unlike Connected Apps, External Client Apps don’t return an error when consumerKey or consumerSecret values are provided. Instead, Salesforce generates new credentials during creation. I agree this behavior isn’t obvious from the current Metadata API documentation. I’ve shared your feedback internally with the product team for consideration. For now, the recommended approach is to use the generated credentials after deployment or use a Packaged External Client App as described above.</em></blockquote><p>Support confirms this is expected, offers a workaround that (below) doesn’t actually work for scratch orgs, and acknowledges the silent-override behavior is a documentation/UX gap merely “shared with product” — not fixed.</p><h4>Problem 2: The recommended workaround doesn’t work on scratch orgs</h4><p>Support’s fix is “use a Packaged External Client App,” with scratch orgs referencing it via orgScopedExternalApp. Deploying (or creating) a Packaged-type ECA against a scratch org fails outright:</p><pre>─────────────── Deploying Metadata ───────────────<br>Deploying v66.0 metadata to test-mtip4uvqwly3@example.com using the v67.0 SOAP API.<br>✔ Preparing 259ms<br>✔ Waiting for the org to respond 5.23s<br>✘ Deploying Metadata 1m 16.72s<br>  ▸ Components: 711/712 (100%)</pre><pre>Component Failures [1]<br>Type: ExtlClntAppGlobalOauthSettings<br>Name: StagingUserConnection_glbloauth<br>Problem: Ephemeral orgs such as scratch orgs can&#39;t be used to deploy packaged ECAs.</pre><p>The officially recommended path is structurally unavailable on the exact org type most SFDX package-development workflows depend on.</p><h4>Problem 3: The ECA is pinned to the Dev Hub it was created on</h4><p>Related, and easy to hit if your team has more than one Dev Hub (e.g. one per product line, or a legacy vs. current Dev Hub): building a package version that references the ECA fails if you’re not on the Dev Hub where the ECA actually lives.</p><pre>sf package version create -d ECATest -x -w 35791 -c<br> ›   Warning: @salesforce/cli update available from 2.140.6 to 2.142.7.<br>Version create.... Create version status: Error<br>Error (MultipleErrorsError): Multiple errors occurred:<br>(1) Filename:package.xml:Unable to retrieve file for id 0xIcb00000076n3 of type ExternalClientApplication.<br>External client apps that are created in ephemeral orgs can&#39;t be packaged.</pre><p>Note the wording: it’s not just “created in a scratch org” — the message treats non-source Dev Hub/packaging orgs the same as ephemeral orgs for this purpose. Practically, this means the ECA and the package version referencing it are tied to a single Dev Hub, with no supported way to move or re-parent that relationship.</p><h4>Problem 4: Omit the OAuth settings and the ECA becomes unusable</h4><p>Sidestep problems 1–3 by excluding ExtlClntAppGlobalOauthSettings from the deploy, and you can&#39;t even open the app in the UI:</p><pre>Failure!<br>We couldn&#39;t load OAuth plugin. Cannot invoke &quot;String.split(String)&quot; because &quot;oauthLink&quot; is null</pre><p>A null-pointer-style failure surfacing straight from Salesforce’s own UI.</p><h4>Net effect: there is no clean path today</h4><p>To develop a package containing an ECA, you need the unpackaged ECA metadata deployed everywhere developers work. But:</p><ul><li>Deploying it regenerates credentials, breaking real integrations, with no error surfaced — a regression versus legacy Connected Apps, which fail loudly on the same scenario.</li><li>The suggested fix (Packaged ECA + orgScopedExternalApp) can&#39;t be deployed to scratch orgs at all.</li><li>Building a package version referencing the ECA fails unless you’re on the exact Dev Hub where the ECA was created.</li><li>Dropping the OAuth metadata to avoid the above makes the ECA non-functional in the UI.</li></ul><p>Compare this to the Connected App era, where a multi-developer team could freely use any combination of:</p><ul><li>Developer Edition orgs</li><li>Partner Developer Edition orgs</li><li>Scratch orgs</li><li>Sandboxes</li></ul><p>With ECA, the only environment that might work cleanly is a <strong>sandbox refreshed from the specific production org where the ECA was originally created</strong> — Support states sandboxes inherit the source org’s Consumer Key/Secret. Not fully validated end-to-end yet, and it still doesn’t help scratch-org-based development or multi-Dev-Hub packaging at all.</p><h4>What teams are doing instead</h4><ul><li>Duplicate the ECA definition into a <strong>separate installable package</strong>, install it on every scratch org / sandbox / DE / PDE org that needs it.</li><li><strong>Remove the ECA metadata from the unpackaged source</strong> used for local package development, so it doesn’t collide with the installed copy.</li></ul><p>Not a supported migration path — a workaround for a workaround. It adds a second package purely to compensate for metadata deployment limitations, and breaks the assumption that unpackaged source is the single source of truth for this piece of metadata.</p><h4>Takeaway</h4><p>If you maintain a managed package with an OAuth/Connected App integration, don’t migrate it to External Client Apps yet — not until Salesforce makes deployed consumerKey/consumerSecret values actually stick, allows Packaged ECAs to be created/deployed on scratch orgs, and decouples package versioning from a single fixed Dev Hub. Right now, none of those are true, and Support&#39;s own response confirms the gaps are known and unresolved.</p><p>If anyone’s found an actual supported path through this — not a Frankenstein workaround — I’d genuinely like to hear it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c67e1f234520" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lightning Web Security setting changes behavior of custom (non-published) attributes on…]]></title>
            <link>https://medium.com/@julfy/lightning-web-security-setting-changes-behavior-of-custom-non-published-attributes-on-c12163bf2a7d?source=rss-2b9cd0ab5025------2</link>
            <guid isPermaLink="false">https://medium.com/p/c12163bf2a7d</guid>
            <category><![CDATA[web-security]]></category>
            <category><![CDATA[salesforce]]></category>
            <category><![CDATA[salesforce-lightning]]></category>
            <category><![CDATA[lightning-web-component]]></category>
            <dc:creator><![CDATA[Julfy Patlatus]]></dc:creator>
            <pubDate>Mon, 06 Jul 2026 08:39:30 GMT</pubDate>
            <atom:updated>2026-07-06T09:22:22.469Z</atom:updated>
            <content:encoded><![CDATA[<h3><a href="https://salesforce.stackexchange.com/questions/439608/lightning-web-security-setting-changes-behavior-of-custom-non-published-attrib">Lightning Web Security setting changes behavior of custom (non-published) attributes on lightning-icon base component</a></h3><p>The same code of LWC Html</p><pre>&lt;template&gt;<br>        &lt;div&gt;<br>            &lt;lightning-icon icon-name=&quot;utility:favorite&quot; value=&quot;1&quot; onclick={handleIconClick}&gt;&lt;/lightning-icon&gt;<br>            &lt;lightning-icon icon-name=&quot;utility:favorite&quot; value=&quot;2&quot; onclick={handleIconClick}&gt;&lt;/lightning-icon&gt;<br>            &lt;lightning-icon icon-name=&quot;utility:favorite&quot; value=&quot;3&quot; onclick={handleIconClick}&gt;&lt;/lightning-icon&gt;<br>        &lt;/div&gt;<br>        &lt;p&gt;Clicked value: {clickedIconValue}&lt;/p&gt;<br>&lt;/template&gt;</pre><p>and Javascript</p><pre>import { LightningElement } from &#39;lwc&#39;;<br><br>export default class WebSecurity extends LightningElement {<br>    clickedIconValue;<br>    handleIconClick(event) {<br>        this.clickedIconValue = event.target.value;<br>    }<br>}</pre><p>works differently in two different orgs, what is the difference?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/222/1*uh6i-qdsUBgeWpno6wyYuA.png" /></figure><p>Both orgs are on the same Salesforce release.</p><p>In org A, when user clicks on a button, the button number is displayed below.</p><p>In org B, when user clicks on a button, nothing happens.</p><p>The difference is in the Web Security settings.</p><p>In org A, Lightning Web Security is turned on and custom undefined attributes are set and passed correctly in DOM.</p><p>In org B, Lightning Web Security is turned off and any custom undefined attributes are ignored and removed from DOM.</p><p>Opening a SF Support Case on this topic.</p><p>Answer from SF Support.</p><p>Kind;y be informed that ‘event.target.value’ behaves differently based on how each security framework handles the DOM at an architectural level.</p><p>Lightning Locker:<br> — — — — — — — — — -</p><p>Lightning Locker secures the DOM by wrapping elements in secure proxy objects such as SecureElement, SecureTemplate, and SecureWindow. When your click handler is called, event.target is one of these proxy objects, and not the real DOM element. The proxy only exposes properties that are part of the component’s officially published API. The published properties of lightning-icon are:</p><p>alternative-text<br>icon-name<br>size<br>src<br>variant</p><p>Since ‘value’ is not a published property, the SecureElement proxy does not expose it, and event.target.value returns undefined. No error is thrown, which can make this behavior difficult to detect during testing.</p><p>Lightning Web Security (LWS):<br> — — — — — — — — — — — — — — — — -</p><p>Lightning Web Security (LWS) takes a fundamentally different approach. Instead of wrapping DOM elements in proxy objects, it uses native browser-level JavaScript isolation. Because the code interacts more directly with the actual DOM element rather than a proxy wrapper, non-published attributes defined in markup may appear accessible as standard DOM properties. This is why event.target.value would work under LWS -&gt; not because value is a supported or an officially exposed property, but because LWS does not filter out such properties in the same way Locker does.</p><p>Kind Note: Even though event.target.value may work under LWS, this is not a supported or guaranteed behavior in either framework. Since value is not a published property of lightning-icon, relying on it is considered undefined behavior, regardless of whether Lightning Locker or LWS is enabled.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c12163bf2a7d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I spent weeks debugging an obscure Salesforce internal error that made no sense.]]></title>
            <link>https://medium.com/@julfy/i-spent-weeks-debugging-an-obscure-salesforce-internal-error-that-made-no-sense-dd820e651ca6?source=rss-2b9cd0ab5025------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd820e651ca6</guid>
            <category><![CDATA[soql]]></category>
            <category><![CDATA[art]]></category>
            <category><![CDATA[atl]]></category>
            <category><![CDATA[query]]></category>
            <category><![CDATA[salesforce]]></category>
            <dc:creator><![CDATA[Julfy Patlatus]]></dc:creator>
            <pubDate>Wed, 01 Jul 2026 11:02:45 GMT</pubDate>
            <atom:updated>2026-07-01T11:02:45.437Z</atom:updated>
            <content:encoded><![CDATA[<p>I spent weeks debugging an obscure Salesforce internal error that made no sense.</p><p>System.QueryException: Aggregate query has too many rows for direct assignment, use FOR loop</p><p>The query had 5 child records with a Long Text Area field and a few hundred records in a completely separate subquery with only Id selected.</p><p>No LTA. No RTA. Just IDs.</p><p>Yet the query was failing.</p><p>After opening a case with Salesforce Support and escalating to the Product Team, I got the official answer:</p><p>“Working as Designed.”</p><p>The Apex runtime applies a shared CLOB budget across ALL child subqueries — even ones with no LTA/RTA fields. The reasoning: heap safety across the entire query tree.</p><p>I wasn’t satisfied with that answer. So I ran dozens of tests across different field sizes and record counts and reverse-engineered the internal formula.</p><p>Here’s what the runtime actually does:</p><p>Step 1 — sum up all declared LTA/RTA field lengths anywhere in the query (parent + every subquery)</p><p>Step 2 — derive a total children row budget:</p><p>FLOOR( 1000 / POW(2, MAX( CEILING(LOG2(totalClobLength / 32768)), 0 )) )</p><p>Which produces this table:</p><figure><img alt="totalClobLength | Row budget — — — — — — — — 256 B — 32 KB | 1,000 32–64 KB | 500 64–128 KB | 250 128–256 KB | 125 256–512 KB | 62" src="https://cdn-images-1.medium.com/max/644/1*G0A1L2o1xv-edZQYXVdEoQ.png" /></figure><p>Both empirical thresholds I observed matched this formula exactly.</p><p>Here’s the flaw.</p><p>The row budget is calculated from LTA/RTA field sizes — and then applied to ALL child records across ALL subqueries, including those that select only Id.</p><p>A subquery like (SELECT Id FROM Children2__r) contributes zero bytes of CLOB content.</p><p>But each of its records still consumes one slot from the budget.</p><p>The budget is asking “how many rows can we safely load given the CLOB volume?”</p><p>Then it counts rows that carry no CLOB data at all toward that answer.</p><p>The fix is straightforward:</p><p>Apply the CLOB row budget only to subqueries that actually select LTA/RTA fields.</p><p>Records in (SELECT Id FROM …) should not count against a heap limit derived from text field sizes.</p><p>I’ve submitted this to IdeaExchange:</p><p>“Fix Design Flaw in internal totalClobLength calculation”</p><p>If you’ve ever hit this error in a multi-relationship query — or you work with contracts, documents, or any domain with rich-text fields on parent objects — please vote.</p><p><a href="https://ideas.salesforce.com/s/idea/a0BHp000017Jka1MAC/fix-design-flaw-in-internal-totalcloblength-calculation">https://ideas.salesforce.com/s/idea/a0BHp000017Jka1MAC/fix-design-flaw-in-internal-totalcloblength-calculation</a></p><p>#Salesforce #Apex #SalesforceArchitect #SalesforceDeveloper #GovernorLimits</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd820e651ca6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Cloud Ingestion API Deployment]]></title>
            <link>https://medium.com/@julfy/data-cloud-ingestion-api-deployment-9e97fd50a297?source=rss-2b9cd0ab5025------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e97fd50a297</guid>
            <category><![CDATA[ingestion-api]]></category>
            <category><![CDATA[data-ingestion]]></category>
            <category><![CDATA[cloud-data]]></category>
            <category><![CDATA[cdp]]></category>
            <category><![CDATA[deployment]]></category>
            <dc:creator><![CDATA[Julfy Patlatus]]></dc:creator>
            <pubDate>Tue, 14 Jan 2025 15:06:15 GMT</pubDate>
            <atom:updated>2025-01-14T15:06:35.162Z</atom:updated>
            <content:encoded><![CDATA[<p>Recently I have been working on a POC involved Data Cloud investigation.</p><p>We have investigated Ingestion API and how it can be deployed and used for data ingestion.</p><p><strong>What is Data Cloud?</strong></p><p>A data platform that unifies all of your company’s data on to Salesforce’s Data Cloud Platform, giving every team a 360-degree view of the customer to drive automation and analytics, personalize engagement, and power trusted AI. Main Data Cloud features are data unification, segmentation, activation.</p><p>However, before data unifications, data must be deduplicated and cleaned on each source system</p><p><strong>Which predefined data connectors are available out of the box?</strong></p><p>Standard predefined Connectors include:</p><ul><li>B2C Commerce Connector</li><li>MC Connectors: Account Engagement, Engagement, Personalization</li><li>SF CRM Orgs Connector</li><li>Some other Web connectors like Amazon S3 or Heroku PostgreSQL</li><li>Mulesoft Anypoint Platform connectors</li></ul><p>However, if you or your client doesn’t have Mulesoft or there is no Mulesoft connector for a given system, you need to build custom Ingestion API connector.<br>We explored possibility of connecting data fromWeInfuse Healthcare system, Google Looker and AdvancedMD. None of these systems are available in standard predefined Web or Mulesoft connectors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/755/1*iNO7trPSuQpyaIQ0sv6OsQ.png" /><figcaption>Data Stream Connectors</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*5tyXJPAU0JBLSKhVq1mvCw.png" /><figcaption>Web App Connections</figcaption></figure><p><strong>Which data ingestion option methods are available?</strong></p><p>Even though documentation mentions only two options of Streaming and Batch (or Bulk) Ingestion method, we may also consider the third Hybrid method which combines both methods in a single implementation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wz4VwVVlpO-8JPv4_bDYaQ.png" /><figcaption>Data Ingestion Methods</figcaption></figure><p><strong>What is Ingestion API and how does it work?</strong></p><p>Ingestion API is a REST API and offers two interaction patterns: bulk and streaming. The streaming pattern accepts incremental updates to a dataset as those changes are captured, while the bulk pattern accepts CSV files in cases where data syncs occur periodically. The same data stream can accept data from the streaming and the bulk interaction.</p><p><strong>Which steps need to be taken to setup Ingestion API?</strong></p><ul><li>Agree on the data to be ingested into Data Cloud and successfully build a YAML file to bring data into Data Cloud.</li><li>Set up the Ingestion API connector and share the object endpoint details with developers.</li><li>Create a connected app and align with your developer on the OAuth flow being used to connect to Ingestion API.</li><li>Create the data streams and deploy them.</li><li>Make calls to the object endpoints and start sending data into Data Cloud.</li><li>Map the data to standard Salesforce objects or custom objects.</li></ul><p><strong>What are the migration limitations related to Ingestion API Deployment?</strong></p><p>Salesforce doesn’t recommend metadata migration via Salesforce CLI.</p><p>They recommend the following methods:</p><ul><li>Deployment via UI (Package Manager and Data Kits)</li><li>Deployment via DataKit Connect API</li></ul><p><strong>Is usual deployment via CLI or Metadata API completely impossible or just not recommended by Salesforce?</strong><br>• It is possible but it is more complicated rather than creating unmanaged package since you would have to figure out all dependency to be migrated</p><p>• Still DataKit Connect API is required to actually migrate the Ingestion API Connector and Data Streams</p><p><strong>Which approach would you recommend for Ingestion API deployment?</strong></p><p>I would recommend performing</p><p>1. deploy Data Kit via SFDX CLI and</p><p>2. deploy the Ingestion API via DataKit Connect API</p><p>In such case, you don’t need to worry if package is expired or unavailable if the source org where it has been created, is expired or deleted.</p><p><strong>What is a Data Kit?</strong></p><p>Data kit is a container for Data Cloud metadata configuration. A data kit wraps around all the packageable Data Cloud metadata so that you can create and deploy complete solutions on Data Cloud.</p><p>Since Data Cloud features are hosted in different server domains then Sales Cloud even though they are available through the same Sales Cloud UI, they cannot be directly migrated or deployed via SFDX or Metadata API but rather they require Data Kit containers creation, which then could be packaged in unmanaged packages or deployed via SFDX CLI.</p><p><strong>What is DataKit Connect API?</strong></p><p>This is not an API per se, but rather just a custom Salesforce provided invocable Flow action, which performs unwrapping and actual deployment of Data Kit components to the corresponding Data Cloud server domain.</p><p>Flow name is sfdatakit__DeployDataKitComponents and this flow is available in the orgs where Data Cloud is enabled.</p><p>Also, a standard flow action is available which has a similar input but accepts only single deployment component input. Standard flow action is used inside a custom flow which executes a loop over deployment component inputs and creates log records. Standard flow action doesn’t create log records and only returns BackgroundOperation record Id which can be queried to get deployment status.</p><p><strong>Is DataKit required for successful Data Cloud deployment?</strong></p><p>Again, Data Kit is required to package Data Cloud metadata components.</p><p>Data Kit container might be added to a package or deployed directly via SFDX or Metadata API. However, it is more complicated to do this via SFDX or Metadata API since one has manually figure out all the necessary Data Kit components.</p><p>So Data Kit container can be migrated via package installation or SFDX deployment.</p><p>However, unwrapping Data Kit container and deploying DataKit components require another call to DataKit Connect API, which can be done via curl tool to perform API request calls or Apex to execute invocable action.</p><p><strong>How DataKit Connect API call looks like and which endpoint does it use?</strong></p><p>This is example of DataKit Connect API call.</p><p>POST <a href="https://{{domain}}/services/data/v%7B%7Bversion%7D%7D/actions/custom/flow/sfdatakit__DeployDataKitComponents">https://{{domain}}/services/data/v{{version}}/actions/custom/flow/sfdatakit__DeployDataKitComponents</a></p><p>Body</p><pre>{<br>  &quot;inputs&quot;: [ { <br>    &quot;dataKitComponentsInput&quot;: [ { <br>      &quot;componentType&quot;: &quot;DataStreamBundle&quot;,<br>      &quot;bundleConfig&quot;: {<br>        &quot;connectorType&quot;: &quot;INGESTAPI&quot;,<br>        &quot;bundleName&quot;: &quot;DataKit&quot;,<br>        &quot;forceNoRefresh&quot;: true,<br>        &quot;bundleIngestApiConfig&quot;: {<br>          &quot;connectorName&quot;: &quot;IngestionAPI&quot;<br>        }<br>      }<br>    } ], <br>    &quot;dataKitNameInput&quot;: &quot;DataKit&quot;,<br>    &quot;dataKitDataSpaceInput&quot; : &quot;default&quot;<br>  } ]<br>}</pre><p>Here {{domain}} is your actual Sales Cloud domain like <a href="https://so1736503059791.my.salesforce.com/">https://client.my.salesforce.com/</a></p><p>Here {{version}} is the preferred API Version, 62.0 or later may be used.</p><p>Body sample input assumes we have an Ingestion API connector with a name IngestionAPI and Data Stream Bundle with a name DataKit and Data Kit with the same name.</p><p>How does successful DataKit Connect API response look like?</p><p>Successul DataKit Connect API response looks like following:</p><pre>[{<br>  &quot;actionName&quot;: &quot;sfdatakit__DeployDataKitComponents&quot;,<br>  &quot;errors&quot;: null,<br>  &quot;invocationId&quot;: null,<br>  &quot;isSuccess&quot;: true,<br>  &quot;outputValues&quot;: {<br>    &quot;Flow__InterviewGuid&quot;: &quot;8432f9d40bd06e260a883e57043419462340427-5db9&quot;,<br>    &quot;Flow__InterviewStatus&quot;: &quot;Waiting&quot;<br>  },<br>  &quot;sortOrder&quot;: -1,<br>  &quot;version&quot;: 1<br>}]</pre><p><strong>Can I invoke DataKit Connect API via Apex?</strong></p><p>Yes, you can.</p><p>This is an example how this can be achieved.</p><pre>sfdatakit.DeployComponentInput dci = (sfdatakit.DeployComponentInput) JSON.deserialize(<br>    JSON.serialize(<br>        new Map&lt;String, Object&gt;{<br>            &#39;componentType&#39; =&gt; &#39;DataStreamBundle&#39;,<br>            &#39;bundleConfig&#39; =&gt; new Map&lt;String, Object&gt;{<br>                &#39;connectorType&#39; =&gt; &#39;INGESTAPI&#39;,<br>                &#39;bundleName&#39; =&gt; &#39;DataKit&#39;,<br>                &#39;forceNoRefresh&#39; =&gt; true,<br>                &#39;bundleIngestApiConfig&#39; =&gt; new Map&lt;String, Object&gt;{<br>                    &#39;connectorName&#39; =&gt; &#39;IngestionAPI&#39;<br>                }<br>            }<br>         }<br>  ), sfdatakit.DeployComponentInput.class<br>);<br>Invocable.Action action = Invocable.Action.createStandardAction(&#39;dataKitDeployComponentAction&#39;);<br>action.setInvocations(<br>    new List&lt;Map&lt;String, Object&gt;&gt;{<br>        new Map&lt;String, Object&gt;{<br>            &#39;dataKitDeployInput&#39; =&gt; dci,<br>            &#39;dataKitName&#39; =&gt; &#39;DataKit&#39;,<br>            &#39;dataSpaceName&#39; =&gt; &#39;default&#39;<br>        }<br>    }<br>);<br>List&lt;Invocable.Action.Result&gt; results = action.invoke();<br>if (results.size() &gt; 0 &amp;&amp; results[0].isSuccess()) {<br>    System.debug(&#39;Success: &#39; + results[0].getOutputParameters().get(&#39;jobId&#39;));<br>} else {<br>    System.debug(&#39;Fail: &#39; + results[0].getErrors()[0].getMessage());<br>}</pre><p>The Job Id parameter returned by the Standard Action, represents the BackgroundOperation record Id which can be queried by SOQL to derive deployment status.</p><p><strong>How can we troubleshoot DataKit Connect API?</strong></p><p>Execution of Custom Flow Action may fail instantly or return waiting status for the flow interview. To get actual deployment status, a query on DataKitDeploymentLogobject must be used (if custom bulk action is used which creates these records):</p><pre>SELECT Id, BundleName, ComponentName, ComponentTemplateId, ComponentType, <br>DataKitName, DataSpaceName, DeployJob, DeploymentError, DeploymentStatus, <br>FlowInterviewIdentifier, Name, <br>PublisherOrgComponentId, SubscriberOrgComponentId, TemplateVersion<br>FROM DataKitDeploymentLog<br>WHERE FlowInterviewIdentifier = &#39;$figuid&#39;</pre><p>BackgroundOperation object also has limited information and can be used as well. Custom Bulk action doesn’t return directly Id of BackgroundOperation records, only standard one which consumes a single deployment input.</p><p>Here $figuid is Flow Interview Guid, which can be retrieved from a successful DataKit Connect API Response by unwrapping outputValues key and then Flow__InterviewGuid key in the inner JSON object.</p><p><strong>Can we visually represent overall process of Data Cloud Deployment which includes Ingestion API connector and data streams?</strong></p><p>Yes, in the following diagram</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sZo69ZzdopHRKwN-qdfBAg.png" /></figure><p><strong>What are Bulk Ingestion Use Cases?</strong></p><p>Use bulk ingestion when moving large amounts of data on a daily, weekly, or monthly schedule. A few example scenarios are:</p><ul><li>Moving large amounts of data on a daily, weekly, or monthly schedule.</li><li>Legacy systems where you can only export data in off-peak hours.</li><li>A new Data Cloud org that you want to backfill with 30, 60, 90, or X days of data.</li><li>Large scheduled Data transfers</li></ul><p><strong>What are Streaming Ingestion Use Cases?</strong></p><p>Use streaming ingestion for small micro-batches of records being updated in near-real time, such as:</p><ul><li>Source system built on modern streaming architectures.</li><li>Continuous data flow</li><li>Event-driven data architecture</li><li>Change data capture events</li><li>Consuming data from webhooks</li></ul><p><strong>What are Hybrid Ingestion Use Cases?</strong></p><p>Use hybrid ingestion for the following case: assume you have a project where you need both initial load and ongoing integration. Then for the initial load bulk ingestion can be used while for the ongoing integration streaming ingestion can be used, in combination usage of both of these approaches yields hybrid ingestion implementation.</p><p><strong>Can we structure differences between bulk and streaming processing as a table?</strong></p><p>Yes, we can structure those in a following table</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wIBRuVGuteUrsYcPaxmK8g.png" /></figure><p><strong>Can we visually represent overall process of Data Cloud Ingestion?</strong></p><p>Yes, in the following diagram</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/789/1*gCZ1NtsUQMbtHcWZLPCakw.png" /><figcaption>Data Cloud Ingestion Flow</figcaption></figure><p><strong>Which necessary scopes are required for the connected app?</strong></p><p>Connected app must have the following scopes,</p><ul><li>Access and manage your Data Cloud Ingestion API data (cdp_ingest_api).</li><li>Access and manage your data (api).</li><li>Perform requests on your behalf at any time (refresh_token, offline_access).</li></ul><p>However, all other Data Cloud recommended to be added as well.</p><p><strong>How Salesforce Access Token can be obtained to be eligible for Data Cloud Token Exchange?</strong></p><p>There are the following options</p><ul><li>Username-password (the only available option in Mulesoft integration)</li><li>JWT Bearer Flow</li><li>Client Credentials Flow</li></ul><p><strong>How does Username-password flow look like?</strong></p><p>Password flow is considered to be the less secure and <strong><em>not recommended</em></strong> since you would have to transfer back and forth user credentials. However, some tool like Mulesoft Anypoint Platform use this option as the only available option implemented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*X-WlZKCJlOLl5ipS5Yh5yQ.png" /><figcaption>Password flow</figcaption></figure><p><strong>How does JWT Bearer Flow look like?</strong></p><p>JWT Bearer Flow has two parameters in the body, grant_type and assertion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/905/1*Ws3P4Ays-YCs5oakgIu4mw.png" /><figcaption>JWT Bearer Flow</figcaption></figure><p>POST <a href="https://{{domain}}/services/oauth2/token">https://{{domain}}/services/oauth2/token</a></p><p>Body:</p><pre>grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer<br>Assertion=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJpc3MiOiIzTVZHOUFPcDRrYnJpWk9JZmg0eWZPQVlQUjFpYzJqb1JDQkhwZFZtaXBXU05sbXNwT2Vib1VtcWdMdHJIcDlRVFRQMGpWc3g0SHl</pre><p><strong>How to build assertion parameter for JWT Bearer Flow?</strong></p><p>X=Base64({“alg”:”RS256&quot;}).Base64({“iss”: ”{{client_id}}”, “sub”: ”{{user.email}}”, “aud”: “https://&lt;login|test&gt;.salesforce.com|site.force.com/portal”, “exp”: “1333685628 {{Date.now/1000+3600}}”})<br> Y=X. Base64( RSASHA256(X, X509 Certificate attached to Connected App) )</p><p>Here Base64() means encode input parameter as Base64 .</p><p>Here {{user.email}} means user email, but it is mainly used only for Experience Cloud users.</p><p>Here aud parameter may have either login.salesforce.com or test.salesforce.com or Experience Cloud portal link .</p><p>Here exp parameter must indicate when token should expire in seconds from start of epoch, in JavaScript we could just add an hour to the current date time returned like Date.now/1000+3600 . JavaScript returns value in milliseconds, so we need to divide by a thousand and an hour is 3600 seconds.</p><p>Here RSASHA256 means RSASHA256 hash of the X variable using X509 Certificate attached to the connected app.</p><p><strong>How does Client Credentials Flow look like?</strong></p><p>POST <a href="https://{{domain}}/services/oauth2/token">https://{{domain}}/services/oauth2/token</a></p><pre>grant_type=client_credentials <br>client_id={{key}}<br>client_secret={{secret}}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*kxt95gJuTnnzND3JeH5QQA.png" /><figcaption>Client Credentials Flow</figcaption></figure><p>Here {{key}} and {{secret}} are Key and Secret from the Connected App. You can rotate them.</p><p><strong>Which Flow would you recommend?</strong></p><p>I would recommend to use Client Credentials Flow, I have implemented this flow in a github repository for the workshop which I conduct based on this talk and article. It is secure and you can rotate Connected App key and secret to make it even more secure.</p><p>JWT Bearer Token is also secure, but it is much complex in implementation. In tradeoff between usability and security I choose Client Credentials flow as simple to implement and secure.</p><p><strong>How Data Cloud Access Token can be retrieved?</strong></p><p>Sales Cloud Access token must be exchanged to receive a Data Cloud Access Token.</p><p>Endpoint: POST <a href="https://{{domain}}/services/a360/token">https://{{domain}}/services/a360/token</a></p><pre>Content-Type : x-www-form-urlencoded<br>grant_type=urn:salesforce:grant-type:external:cdp<br>subject_token={{sid}}00Dzz000000zzzz!AQEAQL7SvQ1t…<br>subject_token_type=urn:ietf:params:oauth:token-type:access_token</pre><p>Here sid is the sales cloud token, which starts from Org Id and then has an asterisk and then random symbols.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*uxoineckrnhG4exyjL5Vww.png" /><figcaption>Data Cloud Access Token Exchange</figcaption></figure><p>This is how response looks like</p><pre>{<br>  &quot;access_token&quot;: &quot;eyJraWQiOiJDT…&quot;,<br>  &quot;instance_url&quot;: &quot;mnrw8yr-m-ydgyzwg…-mq.c360a.salesforce.com&quot;,<br>  &quot;token_type&quot;: &quot;Bearer&quot;,<br>  &quot;issued_token_type&quot;: &quot;urn:ietf:params:oauth:token-type:jwt&quot;,<br>  &quot;expires_in&quot;: 7180<br>}</pre><p><strong>What is the endpoint for streaming ingestion?</strong></p><p>This is the sample endpoint:</p><p>POST <a href="https://{{datacloud-domain}}/api/v1/ingest/sources/%7B%7Bconnector%7D%7D/%7B%7Bentity_name%7D%7D">https://{{datacloud-domain}}/api/v1/ingest/sources/{{connector}}/{{entity_name}}</a></p><p>Authorization: Bearer Data Cloud Access Token</p><p>Content-type: JSON</p><p>Body: { ”data” : [{ “id”: ”1”, “custom_field_column”:”2”}]}</p><p>Here {{datacloud-domain}} is the special data cloud domain which can be obtained by the instance_url key from the successful Data Cloud Access Token exchange call.</p><p>Here {{connector}} is the name of our Ingestion API connector.</p><p>Here {{entity_name}} is the name for the Data Lake Object</p><p>Body contains sample body data, it must have data json key, which expands into array of objects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*CH2LYxB654N3jv0JnW271w.png" /><figcaption>Example for Streaming Ingestion API execution</figcaption></figure><p>What are endpoints for the bulk ingestion?</p><p>These are endpoints for the bulk ingestion. As for any other Bulk API, you need to create a job, post CSV content to batch and then patch the job with UploadComplete state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*32a_e-WsvwScpRLqDFOBMw.png" /><figcaption>Bulk Ingestion Endpoints</figcaption></figure><p><strong>What is covered in the workshop you mentioned before?</strong></p><p>The workshop is intended for Salesforce Partners only or those who have Data Cloud orgs.</p><p>Preliminary steps include the following:</p><ul><li>Get access to the partner learning camp if you don’t have yet</li><li>Create temporary Data Cloud Partner Org from Partner Learning Camp</li><li>Clone provided repository, verify that jq, sfdx, sed, curl are installed</li></ul><p>On the workshop itself, the following steps will be covered:</p><ul><li>Authorize to the org in SFDX</li><li>Launch setup bash script for Connected App and Integration User setup</li><li>Enable Data Cloud Settings manually</li><li>Deploy Ingestion API Connector and Data Streams</li><li>Unwrap data kit components</li><li>Verify DataKit Connect API Deployment status in terminal</li><li>Manually verify client credentials integration user settings</li><li>Launch streaming ingestion bash script</li><li>Launch bulk ingestion bash script</li></ul><p>Connect with me in LinkedIn if you would like me to conduct my workshop in your area.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e97fd50a297" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction Presentation Slide]]></title>
            <link>https://medium.com/@julfy/introduction-presentation-slide-90f334efee53?source=rss-2b9cd0ab5025------2</link>
            <guid isPermaLink="false">https://medium.com/p/90f334efee53</guid>
            <category><![CDATA[professional]]></category>
            <category><![CDATA[introduction]]></category>
            <category><![CDATA[description]]></category>
            <category><![CDATA[presentations]]></category>
            <dc:creator><![CDATA[Julfy Patlatus]]></dc:creator>
            <pubDate>Fri, 22 Nov 2024 14:07:32 GMT</pubDate>
            <atom:updated>2024-11-22T14:07:32.841Z</atom:updated>
            <content:encoded><![CDATA[<p>I am from Ukraine but now travelling around Europe in 2024</p><p>I have currently a position of Associate Salesforce Architect<br>I have 3 personal Listings Published on AppExchange<br>I am the #44th on Salesforce Stack Exchange<br>I am 19x Salesforce Certified Specialist<br>I have 14 years of Development experience<br>I have 12 years of Salesforce Development experience</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bhUKj1jUhn2i_W3Zg7HJHw.jpeg" /><figcaption>Introduction Presentation Slide</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=90f334efee53" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prerequisites for MuleSoft Data Cloud Integration]]></title>
            <link>https://medium.com/@julfy/prerequisites-for-mulesoft-data-cloud-integration-ca519c08f478?source=rss-2b9cd0ab5025------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca519c08f478</guid>
            <category><![CDATA[stream]]></category>
            <category><![CDATA[cloud-data]]></category>
            <category><![CDATA[mulesoft]]></category>
            <category><![CDATA[integration]]></category>
            <dc:creator><![CDATA[Julfy Patlatus]]></dc:creator>
            <pubDate>Mon, 11 Nov 2024 10:19:28 GMT</pubDate>
            <atom:updated>2024-11-11T10:19:28.488Z</atom:updated>
            <content:encoded><![CDATA[<p>Recently I have discovered that my blog on wordpress has been deactivated by some mistake so I was suggested to start blogging on medium.</p><p>Let’s start with some short note about MuleSoft Data Cloud integration.</p><p>Before Mulesoft can send data into Data Cloud, the following actions should be performed on the target organization</p><p>Create Connected App with the following scope<br>- Access Interaction API Resources (interaction_api)<br>- Access all Data Cloud API Resources (cdp_api)<br>- Manage DC CI data<br>- Manage DC IR<br>- Manage DC Ingestion API data<br>- Manage DC profile data<br>- Manage user data<br>- Perform ANSI SQL queries on DC<br>- Perform segmentation on DC data</p><p>Enable OAuth Settings in Connected App<br>Enable Client Credentials Flow</p><p>Copy consumer key, secret key and provide to Mulesoft team.</p><p>Set OAuth and OpenId connect settings</p><p>[v] Allow OAuth Username-Password Flows<br>[v] Allow OAuth User-Agent Flows</p><p>Ingestion API<br>- New: My Ingestion API. Upload schema. Should be prepared in YAML format and discussed with MuleSoft team.<br>The status should be “Needs a data stream”</p><p>Go to<br>- Data Cloud/ Data Streams/New Data Stream/Ingestion API.</p><p>and status should be changed to “In Use”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca519c08f478" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>