Prerequisites

Sitecore relies on native Windows IFilters to extract text from media documents. To index PDFs successfully:

• Adobe PDF IFilter Version 9 is Required: You must install the Adobe PDF IFilter version 9 (or later 64-bit compatible version for modern servers) on every Content Management (CM) and Content Delivery (CD) server that handles indexing.
• Download Location: The necessary IFilter files are often available via Adobe’s legacy FTP server: ftp://ftp.adobe.com/pub/adobe/acrobat/win/9.x/. Ensure you choose the correct 32-bit or 64-bit version matching your server’s architecture and IIS Application Pool settings. Since for me the above link didn’t worked so I downloaded from https://community.adobe.com/t5/acrobat-discussions/current-official-download-link-for-adobe-pdf-ifilter-11-for-64-bit-platforms/td-p/11988379
• Permissions: The IIS Application Pool identity running your Sitecore instance must have read permissions to the IFilter installation directory and the Sitecore media library folder.

Configuration Steps

Follow these steps in order to enable, configure, and verify PDF content indexing.

1. Install the Adobe PDF IFilter

Download and install the Adobe PDF IFilter software on all relevant Sitecore servers. This is a system-level installation.

2. Configure Sitecore for IFilter Usage

You need to tell Sitecore to use the Windows IFilter service instead of its default, basic media extractor. This is done in the Sitecore.ContentSearch.ContentExtraction.config file (located in App_Config\Sitecore\ContentSearch\) or via a patch file.

Modify the <mediaFileTextExtractor> node:

<configuration xmlns:patch="www.sitecore.net">
  <sitecore>
    <contentSearch>
      <contentExtraction>
        <mediaFileTextExtractor type="Sitecore.ContentSearch.ContentExtraction.IFilter.IFilterMediaFileTextExtractor, Sitecore.ContentSearch.ContentExtraction"/>
      </contentExtraction>
    </contentSearch>
  </sitecore>
</configuration>

3. Verify File Types and MIME Types for Indexing

Ensure that the PDF MIME type (application/pdf) is explicitly allowed within the index configuration.

Check the <mediaIndexing> section within your Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config (or a custom patch file) to confirm that PDFs are included and not excluded.

4. Ensure the Content Field is Mapped Correctly in Sitecore

This is a critical step to ensure the data is stored and visible in the Solr Admin UI. The default Sitecore configuration often uses returnType="string", which prevents the large text blob from being stored in a viewable manner.

You must change returnType="string" to returnType="text".

In your configuration patch file, the computed field definition should look exactly like this:

<fields hint="raw:AddComputedIndexField">
  <!-- Ensure returnType="text" is used for large content extraction -->
  <field fieldName="_content" returnType="text" type="Sitecore.ContentSearch.ComputedFields.MediaItemContentExtractor,Sitecore.ContentSearch">
    <defaultMediaExtractor ref="contentSearch/indexConfigurations/contentExtraction/defaultMediaExtractor"/>
    <mediaIndexing ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration/mediaIndexing"/>
  </field>
</fields>

5. Verify the Solr Schema Field Definitionmanaged-schema

While Sitecore maps the field dynamically, you need to ensure the target Solr field has stored="true".

• Access your Solr Admin Panel (e.g., http://localhost:8983/solr/) and select your index core (e.g., sitecore_master_index).
• Navigate to the Schema section.
• The _content field (or the dynamically mapped field like _content_t if using dynamic fields) must be configured as stored="true". A typical working definition looks like this:

<field name="_content" type="text_general" indexed="true" stored="true" multiValued="true"/>
<!-- Or if using dynamic fields, the template should support stored="true" -->

6. Rebuild the Search Indexes

Configuration changes require a full index rebuild to apply the new schema mapping and re-extract the media content using the IFilter.

• In the Sitecore Launchpad, open the Control Panel.
• Click on Indexing Manager.
• Select the relevant indexes (e.g., sitecore_master_index, sitecore_web_index) and click Rebuild.

As shown in below screenshot, after doing above configuration changes, I was able to see my file content appearing in _content_t field.

Since I modified/checking on sitecore_master_index, I was able to verify the same thing in CMS search bar by searching for a text which is part of pdf’s content.

Debugging and Verification

If you followed all steps and still don’t see the content, use the Solr Admin panel to debug the results directly.

1. Go to the Query Tab in the Solr Admin UI for your core.
2. Query for a specific PDF item by its Sitecore GUID using the _id field (e.g., q=_id:"{YOUR-ITEM-GUID}").
3. Execute the Query and inspect the resulting JSON/XML output. The _content field should now be present and populated with the text extracted from your PDF document.

In this blog, we’ll walk through:

1. How to create a Search Results widget in the Sitecore Search Customer Engagement Console (CEC)
2. Configure its settings (RFK ID, variation, rules)
3. Build a React widget component (using the Search JS SDK + UI components)
4. Integrate it into a Next.js app

By the end, you’ll have a working SearchResult widget that’s configured in Search UI and rendered in your React frontend.

Step 1: Create the Widget in Sitecore Search CEC

• Log into the Customer Engagement Console (CEC) for your Sitecore Search instance.
• Navigate to the Widgets section.

• Click Add widget → Search results.(For this blog we are using search results widget)

1. In the configuration screen:

• Enter Widget Name
• Provide a RFK ID (this is the unique ID you’ll use in your frontend code)
• Set the Variation name (by default there’s “Default”)
• Choose whether the widget “Will be used in” a form / page, etc.(The options are: Page, Dialog, and Common across all pages. Select “Common across all pages” as we want to implement this widget on every page of our site.)

• Click Next, review settings, then Save.

Step 2: Configuring Rules For Widget

Once you have created widget, click on it and you will see the screen as below where you see variations listed(atleast one default variation is mandatory and available by default as you create widget).

Click on it and it will take you to the rules section, where clicking on “Add Rule” will open another screen where you can add configuration rules and settings for the widget.

• After saving, click Publish so the widget is live.

Step 3: Integrate Search in Application

Considering you already have a Next.js running application, now we need to install the packages needed for the Search JS SDK to integrate Sitecore Search with our Next.js app.

Open the terminal and run the following commands:

npm install --save @sitecore-search/react
npm install --save @sitecore-search/ui
npm install --save-dev @sitecore-search/cli

We will make use of storybook documentation so download or copy the component from there and create widget(https://developers.sitecorecloud.io/search-sdk/react/latest/storybook/index.html?path=/story/widget-templates-content-search-results-basic--page)

In your root level file(ex- i added in layout.tsx) import PageController and add following to set our locale to “en-US”:

 
import { PageController} from "@sitecore-search/react";

PageController.getContext().setLocaleLanguage('en');
PageController.getContext().setLocaleCountry('us');

Now at your component level, import your widget which you created above and import WidgetProvider

 
import {WidgetsProvider } from "@sitecore-search/react";

Next, wrap the main <Component> inside of a new ‘WidgetsProvider’:

<WidgetsProvider
      env={process.env.SEARCH_ENV}
      customerKey={process.env.SEARCH_CUSTOMER_KEY}
      apiKey={process.env.SEARCH_API_KEY}
    >
    <Component {...pageProps} />
  </WidgetsProvider>

Test in your browser:

• Enter a search query
• See if results appear
• Check network calls to verify correct rfk_id is being used

Conclusion

Creating a SearchResult widget in Sitecore Search (via the CEC portal) and integrating it into your React/Next.js frontend using the Search SDK gives you a powerful, maintainable search experience.

Sitecore’s headless ecosystem has evolved significantly with the rise of XM Cloud. Today, developers often face a common question:

Should we use JSS or Content SDK for our Headless implementation?

Both tools enable modern front-end development with Angular, React, Next.js, Vue, etc. But their architecture, purpose, compatibility, and use-cases are completely different.

In this blog we will break it down in the simplest possible way.

What Is JSS?

JSS (JavaScript Services) is Sitecore’s original headless development framework designed for XM, XP, XM Cloud.

It delivers full layout information, component mapping, and routing using Sitecore’s Layout Service.

Key Features of JSS

• Component-based architecture
• Layout Service (JSON output of page + components)
• Sitecore-managed routing
• Experience Editor support
• Personalization support
• Headless SXA support
• Supports React, Angular, Vue

JSS SDK — Layout Service JSON (Full Page + Components)

This is what JSS returns when you request a page, e.g.:

/sitecore/api/layout/render/jss?item=/home&sc_lang=en

{
  "sitecore": {
    "context": {
      "pageEditing": false,
      "language": "en",
      "site": {
        "name": "jss-app"
      }
    },
    "route": {
      "name": "home",
      "displayName": "Home",
      "fields": {
        "pageTitle": {
          "value": "Welcome to Our Website"
        },
        "heroImage": {
          "value": "/-/media/hero.jpg"
        }
      },
      "placeholders": {
        "main": [
          {
            "componentName": "HeroBanner",
            "fields": {
              "title": { "value": "Hello from JSS" },
              "subtitle": { "value": "Hero banner from Layout service" }
            }
          },
          {
            "componentName": "TextBlock",
            "fields": {
              "text": { "value": "Text block from layout service" }
            }
          }
        ]
      }
    }
  }
}

What Is Content SDK?

Content SDK is XM Cloud’s pure content API SDK.
It is used when you want only content data, not Sitecore layouts or renderings.

It communicates with Sitecore Edge (GraphQL APIs).

Key Features of Content SDK

• Raw content delivery (no renderings)
• Edge GraphQL queries
• Fully decoupled front-end
• No Experience Editor
• No Layout Service
• No Sitecore routing
• Fast and cloud-native
• Works only with XM Cloud (not XP/XM classic)

Content SDK — Raw Content JSON (GraphQL Query Response)

This is what you get from XM Cloud Content Delivery / Edge:

query {
  item(path: "/home", language: "en") {
    name
    url
    pageTitle: field(name: "pageTitle") { value }
    heroImage: field(name: "heroImage") { jsonValue }
    components: children {
      edges {
        node {
          name
          template {
            name
          }
          fields {
            name
            jsonValue
          }
        }
      }
    }
  }
}

{
  "data": {
    "item": {
      "name": "home",
      "url": "/home",
      "pageTitle": { "value": "Welcome to Our Website" },
      "heroImage": {
        "jsonValue": {
          "src": "/-/media/hero.jpg",
          "alt": "Hero Image"
        }
      },
      "components": {
        "edges": [
          {
            "node": {
              "name": "herobanner",
              "template": { "name": "HeroBanner" },
              "fields": [
                {
                  "name": "title",
                  "jsonValue": "Hello from Content SDK"
                },
                {
                  "name": "subtitle",
                  "jsonValue": "This is content from GraphQL"
                }
              ]
            }
          },
          {
            "node": {
              "name": "textblock",
              "template": { "name": "TextBlock" },
              "fields": [
                {
                  "name": "text",
                  "jsonValue": "This is a content block"
                }
              ]
            }
          }
        ]
      }
    }
  }
}

Architecture Difference

JSS Architecture (Layout Service Based)

Used for traditional headless rendering with Sitecore-managed layout.

Flow:

1. Frontend (React/Angular/Next.js) requests a route.
2. Layout Service returns: Component list, Placeholders, Data source items, Page metadata
3. Rendering Host renders each component.
4. Experience Editor + personalization are enabled.

Content SDK Architecture (Pure Content API)

Used for content-driven experiences.

Flow:

1. Frontend queries Sitecore Edge using GraphQL.
2. Response contains raw data only.
3. Frontend fully controls:

• Routing
• Component structure
• Page layout

Compatibility Matrix (XM, XP, XM Cloud)

When Should You Use JSS?

Use JSS when:

✔ You need Experience Editor
✔ You need component editing
✔ You want Sitecore-managed routing
✔ You’re using personalization
✔ You want start from Sitecore templates + renderings
✔ You’re building a component-heavy corporate website
✔ You need multisite + multilingual with SXA headless

When Should You Use Content SDK?

Use Content SDK when:

✔ You need raw content only
✔ You want front-end fully in control
✔ You don’t need layout service
✔ You don’t use renderings / placeholders

Conclusion

Both JSS and Content SDK are powerful — but serve totally different purposes:

• JSS = Sitecore-driven components + layout (best for websites)
• Content SDK = pure content API (best for apps and developer-driven sites)

Choosing the right one depends on your needs around personalization, Experience Editor, routing, and layout control.

What is Sitecore AI?

Sitecore AI is a new AI-driven experience layer that integrates with Sitecore’s SaaS and composable products — XM Cloud, CDP & Personalize, Search, and Content Hub.

It’s designed to bring agent-powered personalization, intelligent automation, and AI-assisted content creation into every part of the customer journey.

Why Sitecore Introduced Sitecore AI

Digital experience platforms are becoming increasingly complex. Marketers and developers need:

• Faster content operations
• Automated personalization at scale
• Intelligent insights without heavy manual analysis
• Better orchestration between content, customer data, and experiences

Sitecore AI is the answer — transforming Sitecore into an “agentic” DXP, where AI agents help plan, create, optimize, and personalize experiences automatically.

Key Capabilities (Current & Emerging)

1. AI-Driven Content Creation

Helps teams generate content variants, summaries, metadata, translations, and creative assets faster and more consistently.

2. Automated Personalization

AI can analyze behavior, segments, and intent, and deliver the right content automatically — reducing the manual setup required for personalization campaigns.

3. Intelligent Search Optimization

By integrating with Sitecore Search, AI improves relevance ranking, query understanding, recommendations, and zero-result handling.

4. Customer Insights & Predictive Intelligence

Combined with CDP & Personalize, Sitecore AI will bring stronger predictive propensity models, journey predictions, and behavioral clustering.

5. Agentic Automation (Future-focused)

Sitecore AI will introduce AI Agents that can assist with tasks such as:

• Creating personalized experiences
• Running optimization experiments
• Identifying content gaps
• Suggesting new pages or components
• Automating workflows

How Sitecore AI Fits Into Existing Products

Today:

• AI enhances features inside XM Cloud, Search, and CDP.
• Some capabilities are in early access or preview.

Tomorrow:

• Sitecore AI becomes the central intelligence layer across all Sitecore tools.
• Expect deeper integration with Content Hub, XM Cloud components, design systems, and personalization engines.

In summary:
It’s not replacing XM, XP, CDP, or Search — it is powering them.

Why It Matters for Developers & Marketers

For Developers

• Better APIs and automation tooling
• AI-powered components and experience optimization
• Faster delivery cycles
• Less manual personalization logic

For Marketers

• Create content faster
• Launch experiences with AI-recommended variants
• Personalize without needing deep technical knowledge
• Act on insights instead of manually analyzing data

Conclusion

Sitecore AI marks the beginning of a new era for Sitecore’s digital experience ecosystem.
While still evolving, its direction is clear: a unified AI layer that makes content creation smarter, personalization automatic, and customer journeys more connected.

For Sitecore XP/XM users, Stream isn’t just another bolt-on; it fundamentally transforms content workflows by embedding advanced AI tools directly within the Content Editor and Experience Editor. This means you gain immediate access to powerful capabilities like AI-driven content generation, seamless translation, and efficient content extraction, all precisely where you already work. Crucially, these features are fully configurable to align with your unique brand voice and are robustly connected to your Sitecore instance through Stream’s modern SaaS APIs.

Prerequisites

• Access to sitecore cloud portal(https://portal.sitecorecloud.io/) to download the latest Stream package.
• A valid Sitecore Stream license
• Sitecore XP/XM 10.2 or later.

Step by step configuration:-

1. Login to sitecore cloud portal, navigate to the apps section and then to Stream. click on “Sitecore stream for platform DXP”

It will take you to the developers portal form where you can download latest release of sitecore stream.

Unzip the downloaded package. Inside, you’ll see docs plus a SitecorePackage.zip.

2. Log into your XP/XM instance. Go to Launchpad > Control Panel > Administration > Install a package.

Upload and install the SitecorePackage.zip.

after installing, check your dashboard — there should be a Stream section.

NOTE: After installing package you may see all/some features as disabled or even clicking on them can give error but that is not an issue. You can continue with next steps.

3. Create API Credentials in Sitecore Cloud Portal

You need a Client ID and Client Secret to connect XP with Stream:

• Log into the Sitecore Cloud Portal. https://portal.sitecorecloud.io/
• Open the Stream app and go to Admin > XP/XM Keys as shown in below screenshot.
• Click create credentials, it will open popup, give some label and description -> create. It will create Client Id and Client Secret.

NOTE: the secret is shown only once so save it.

4. Update Connection Strings

In your Sitecore XP/XM installation, Edit your ConnectionStrings.config file:

C:\inetpub\wwwroot\YourSitecoreCm\App_Config\ConnectionStrings.config

<add name="sitecore.aiclient.stream" connectionString="client_id=YOUR_CLIENT_ID;client_secret=YOUR_CLIENT_SECRET;authority=https://auth.sitecorecloud.io;audience=https://api.sitecorecloud.io" />

Replace clientid and secret with your real values.

some feature not enabled- that is not issue, its based on subscription.

5. Verify the setup

• Open the Content Editor.
• Look for the AI icon beside sitecore item fields.
• Click it and test generating or refining content. If the dialog opens and works, you’re good to go.

I selected my brand kit and asked stream to generate some content based on my query. Worked as shown below:

Errors you may face and their resolution.

1. Unknown connection string ‘sitecore.aiclient.stream’ — reverify the connection string in connectionstring.config file, if details are correct then try to restart application and check again, it should work.

References:

https://doc.sitecore.com/xp/en/users/latest/sitecore-experience-platform/install-and-configure-sitecore-stream-for-platform-dxp.html

Sitecore has been a pioneer in digital experience management — empowering brands to deliver personalized, consistent content at scale. From Sitecore XP to XM Cloud, the platform has evolved toward a fully composable, SaaS-first ecosystem.

Now, Sitecore is taking its biggest step yet into the world of AI with Sitecore Stream — a new AI-powered, SaaS-based platform that redefines how marketers, content creators, and developers collaborate to create brand-aligned digital experiences.

Sitecore Stream is brand-aware AI — meaning it learns your brand’s tone, guidelines, and messaging style so everything it generates feels authentically you.

In this first post of our “Mastering Sitecore Stream” series, we’ll explore:

• What Sitecore Stream is and how it fits into the modern Sitecore ecosystem
• Why it matters for marketers, content ops, and developers
• The core building blocks: Brand Kits, AI Agents, and Agentic Workflows

What Is Sitecore Stream?

Sitecore Stream is an AI layer built into Sitecore’s SaaS ecosystem. It is an AI engine that sits between your content and your experiences. It helps you-

• Generate content faster, grounded in your brand identity
• Automate campaign and workflow tasks
• Improve collaboration between marketing and technical teams
• Maintain brand voice and tone at every touchpoint

Where It Fits in the Sitecore Ecosystem

Sitecore Stream integrates seamlessly with other SaaS offerings through the Sitecore Cloud Portal, acting as a shared intelligence layer:

• For XM Cloud, it powers AI-driven content creation and page assembly.
• For Content Hub, it supports asset generation, metadata tagging, and translations.
• For Personalize, it provides AI-based content variants and suggestions.

Why Sitecore Stream Matters

Today’s marketing teams face immense pressure — multiple channels, multilingual content, shorter go-to-market times, and fewer resources.
Meanwhile, developers are responsible for maintaining flexible, API-driven front ends that can adapt to rapid content changes.

Here’s where Stream makes a difference:

Brand Kits — Teaching AI Who You Are

At the heart of Sitecore Stream lies the Brand Kit — the foundational element that tells the AI what your brand stands for.

What’s Inside a Brand Kit

A Brand Kit is a curated collection of your brand’s assets, documentation, and rules, such as:

• Tone of Voice & Guidelines: Upload your brand style guides or editorial policies.
• Visual Identity: Include sample visuals, logo guidelines, or imagery references.
• Terminology & Vocabulary: Define approved terms and phrases unique to your brand.
• Reference Content: Provide previous campaign materials, blogs, or product copy as examples.

Once uploaded, Stream uses these assets to ground its AI models using Retrieval-Augmented Generation (RAG) — meaning every AI output is based on your brand data, not just a public language model.

Example in Action

You ask Stream:

“Write a product announcement email for our eco-friendly home appliance range.”

Instead of generating a random email, Stream pulls context from your Brand Kit — your tone (“friendly yet expert”), your product vocabulary (“eco-efficient”), and your past campaign style.
The result? A perfectly on-brand draft that feels like it was written by your own marketing team.

AI Agents — Your Brand-Aware Copilots

While Brand Kits teach AI what your brand is, AI Agents (or Copilots) use that knowledge to do work for you.

These are purpose-built assistants embedded into Sitecore Stream’s interface that help different roles — marketers, content authors, and strategists — complete their day-to-day tasks faster and more intelligently.

Key AI Agents in Sitecore Stream

1. Content Copilot

• Drafts blog posts, landing pages, and social copy.
• Suggests variations and translations based on your Brand Kit.
• Refines existing content for tone, SEO, or clarity.

2. Brief Copilot

• Generates marketing or campaign briefs automatically from a few inputs.
• Recommends structure, messaging, and deliverables for campaigns.

3. Experience Copilot

• Helps build or personalize web pages within XM Cloud.
• Suggests layouts, assets, or copy blocks that align with the campaign goal.

Agentic Workflows — Automating the Entire Content Journey

Beyond generating content, Sitecore Stream’s Agentic Workflows bring automation to your marketing operations.

These AI-driven workflows connect tasks, people, and deliverables across the full lifecycle — from ideation to publishing.

What They Do

• Automatically assign tasks (drafting, reviewing, translating).
• Generate campaign deliverables from a single brief.
• Suggest next actions (“Translate this blog into French,” “Generate a short-form version for LinkedIn”).
• Maintain audit trails for governance and compliance.

Here’s how the three pillars interact:

Brand Kits → inform → AI Agents → trigger → Agentic Workflows

1. Brand Kits teach AI how to represent your brand.
2. AI Agents (Copilots) use that intelligence to create on-brand content.
3. Agentic Workflows automate the review and delivery of that content across your DXP.

Getting Started

1. Log in to Sitecore Cloud Portal and enable Sitecore Stream.
2. Create your first Brand Kit — upload your brand docs, style guides, and reference materials.
3. Experiment with a Copilot (start with Content Copilot) to see how it writes using your brand tone.
4. Try an Agentic Workflow — such as a blog creation-to-approval pipeline.
5. Review results, refine your Brand Kit, and iterate.

Conclusion: A Smarter, Brand-Aware DXP

Sitecore Stream represents the future of intelligent content operations — an AI that doesn’t just generate text, but truly understands your brand, your tone, and your audience.

By combining Brand Kits, AI Agents, and Agentic Workflows, Stream empowers teams to move faster, stay consistent, and focus on creativity rather than manual work.

In the next post, we’ll go hands-on with Brand Kits — exploring how to create one, upload brand materials, and experience how Stream turns your brand identity into actionable AI intelligence.

Now, let’s dive into one of the most developer-centric features of Sitecore Search — the Ingestion API.

What is the Ingestion API?

The Ingestion API is a REST-based interface provided by Sitecore Search that allows developers to push, update, or delete content in a search index programmatically.

It’s especially useful when:

• Your content doesn’t come from a crawler-supported source (like a CMS or website).
• You want real-time indexing (e.g., product updates, article publishing, or user-generated content).
• You need to integrate external systems or databases with Sitecore Search.

Instead of relying solely on scheduled crawlers, the Ingestion API gives you direct control over what goes into your index — and when.

When to Use the Ingestion API

Use the Ingestion API when:

• Your data is dynamic.
• Your content is stored outside Sitecore or any CMS (like in ERP, PIM, or external APIs).
• You want incremental updates without re-crawling everything.

Authentication & API Access

To use the Ingestion API, you’ll need:

• Your Base URL
• Domain ID
• Source ID
• Entity ID
• API Key

You can find these under Developer Resources → API Access in your Sitecore Search CEC.

Each API key has one or more scopes.
For ingestion, make sure your key includes the Ingestion scope.

Authentication is done via header:

Authorization: ApiKey {your-api-key}

API Endpoint Format

The Ingestion API exposes multiple endpoints depending on what you want to do.
Below are the commonly used ones.

1. Create or Update a Document

POST /ingestion/v1/domains/{domainId}/sources/{sourceId}/entities/{entityId}/documents?locale={locale}

2. Delete a Document

DELETE /ingestion/v1/domains/{domainId}/sources/{sourceId}/entities/{entityId}/documents/{documentId}?locale={locale}

3. Check Status of an Update

GET /ingestion/v1/domains/{domainId}/sources/{sourceId}/entities/{entityId}/status/{incrementalUpdateId}

Each {domainId}, {sourceId}, {entityId}, and {locale} corresponds to values defined in your Sitecore Search configuration.

Payload Structure

The Ingestion API expects a JSON object that wraps your content inside a document object, which includes:

• A unique id
• A set of fields that match your entity’s schema

Example Payload

{
  "document": {
    "fields": {
      "title": "Flashing News - Sitecore acquires RFK",
      "description": "Hype around the new search technology"
    },
    "id": "document1"
  }
}

• document → root object
• fields → holds all attribute/value pairs defined in your entity schema
• id → unique identifier for that document

Multi-locale Ingestion

If your site supports multiple locales, you can send the same document with a locale parameter — for example:

POST .../documents?locale=en_us
POST .../documents?locale=ar_sa

Each localized document will be treated as a variation of the same entity record.

Monitoring & Troubleshooting

You can monitor ingestion status directly in the CEC:

• Go to Content Collection → your Entity → Content sync section
• View notification about item being created.

Common issues:

• 400 Bad Request → Check field names match your entity schema, unique id is provided.
• 401 Unauthorized → Invalid or missing API key.
• 409 Conflict → Duplicate document ID or locale mismatch.

Note: you must enable incremental updates for a source in the CEC.

Rferences:

• Using the Ingestion API to add content to an index
• Ingestion API

Global Resources in Sitecore Search — Overview

In the Sitecore Search Customer Engagement Console (CEC) , Global Resources is a central area for managing configurations that apply across your search implementation — not just individual widgets or pages.

The Global Resources section typically includes:

1. Global Widget
2. Recipes
3. Synonyms
4. Notifications

Each one plays a distinct role. Below is a deep dive into each.

1. Global Widget

What Is It?

• The Global Widget acts as a base template or foundation for all other widgets in your search experience.
• All “regular” widgets you create (for result listings, recommendations, etc.) will by default inherit settings from the Global Widget unless overridden.
• The Global Widget can have variations, one of which must be marked as the default variation.

What You Can Configure in Global Widget / Its Variations

In each variation, you can define:

• Site / visitor context rules — when this variation should be active (e.g. based on user type, locale, device)
• Merchandising settings — global settings like boosting, burying, pinning rules
• Defaults setting for textual relevance, sort order, personalization, facets, textual relecance, search ranking etc.

2. Recipes

What Are Recipes?

• A Recipe is a preconfigured logic or algorithm that determines which items to recommend in a recommendation widget.
• Recipes define how recommendation logic operates — e.g. “recently viewed items,” “most clicked items in last week,” “most recently added to catalog” etc.
• When creating a recommendation widget, you select one recipe (or a compatible recipe) for the entity (product / content type) you want to recommend.

3. Synonyms

What They Are

• Synonyms let you map keywords or phrases to related terms so that when a user searches for “X,” your system also considers “Y.”
• They help improve recall (find more relevant items) and user experience by dealing with linguistic variability (plurals, alternative words, etc.).

Types of Synonyms / Word Relationships

• One-way synonyms — Map a keyword to additional terms, but not the other way around. E.g. searching “car” returns results for “automobile” too, but searching “automobile” may not return “car” unless you define it.
• Two-way synonyms — Bidirectional equivalence; both terms are treated interchangeably. E.g. “trousers” ↔ “pants.”
• Replacement / substitution — Instead of expanding the search, you replace a term. For example, treat “chevy” as “chevrolet,” so “chevy” searches are mapped to “chevrolet.”

4. Notifications

What They Are

• Notifications is a section within Global Resources that surfaces alerts, messages or operational updates about your search implementation.
• You can view notification history, filter by severity, and see details about occurrences.
• Users can configure subscription preferences for which notifications they want to receive (e.g. via email).

To get started with CEC portal, you should have login credential for CEC poral- https://cec.sitecorecloud.io/.

Navigating the CEC Portal

• Site performance — The CEC homepage: a quick overview of site performance analytics and health.
• Pages — Manage your search pages and their associated widgets that render search experiences.
• Widgets — Create and configure reusable UI components (widgets) that display search results and features.
• Analytics — Access reports and metrics that show how your search implementation is performing.
• Global resources — Configure system-level resources and global widget settings used across pages.
• Content collection — Browse and validate all indexed content and visitor data in your domain.
• Sources — Define and configure content sources (feeds, crawlers, uploads) and their extractors for indexing.
• Developer resources — Tools and API references for developers to test and work with Sitecore Search services.
• Administration — Manage domains, users, roles, and other administrative settings for your tenant.

Types of Sources & Connectors in Sitecore Search

In Sitecore Search, a Source tells the platform where and how to pull in content. Each source type is designed for different content scenarios:

• Web crawler — simple pull crawler that follows links (or a sitemap) and indexes HTML/PDF/Office content; good for single-locale public websites.
• Advanced web crawler — enhanced crawler with domain restrictions, parallel workers, JS rendering, auth, multiple-entity extraction and locale handling (use when site is complex or gated).
• Feed crawler — ingest structured files (CSV/JSON/delimited) or SFTP uploads and transform rows/objects into index documents (use for batch product/catalog feeds).
• API crawler — pulls JSON from API endpoints and uses JSONPath/JS extractors to create documents (use for headless CMS APIs, XM/Edge, or product APIs).
• API push (API Push) — push-only source that creates an empty index you populate via the Ingestion API (ideal for real-time/event-driven content or frequent small updates).

Document extractor

A Connector in CEC is the selected ingestion mechanism (the “type” you pick when creating a Source) — e.g., Web Crawler, Feed Crawler, API Crawler, or API Push. It determines how Search reaches your content (pull vs push) and which extraction/transform features are available.

• XPath — HTML/XML extraction (Web crawler / Advanced web crawler).
• CSS selector — HTML/CSS extraction (Advanced web crawler).
• JavaScript function / Cheerio-style JS — complex/custom extraction or transformations (Advanced web crawler, API crawler, some feed scenarios).
• JSONPath — parse API/JSON payloads (API crawler; also used when extracting from JSON feeds).

Now that we understand sources, connectors, and document extractors, let’s walk through creating a Web Crawler source step by step.

Create and configure your first source

or this blog post, I’ll show you how to create a Web Crawler source, as that is the most common source you’ll use.

Go to sources and click on add source.

Provide source name, connector and short description and save it.

The source will be created and you can see some basic information of the source, like the Source ID, the status and the last run status & time (empty since the source was never run). The source is still in Draft status and the Publish button is disabled, so there is required configuration that needs to be completed before we can publish it.

The source comes with pre-configured attribute extraction rules. The default ones are name, url, image_url, description and type attributes to the Open Graph tags in the HTML markup. In addition to this or if you want you can remove/add/update extraction of value form your specific html tag and update this extraction by clicking on edit button. As I have modified it with some additional attributes and removed default ones.

Next in the Web Crawler Settings section click on edit and enter the URL to your website in the URL field. Also there are additional settings like max depth of url allowed, max no of urls etc confifurations can be done.

Next, we can also set the scan frequency. This will ensure that our website is automatically re-scanned periodically, to ensure that any content updates on our website are correctly crawled and reflected in our search index.

last step, click on the ‘Publish’ button in the top-right corner to publish our new web crawler source.

Once published your crawled content will be available in the content collection. Navigate to content collection, expand your entity under which you have created your source then filter your source from top navigation(by default you will see all the sources’s content). You will be able to see all your crawled pages content mapped to attribute fields.

This confirms that our source is ready to be queried via the Search and Recommendation API, so we can start integrating it with our website.

Sitecore Search is a cloud-native, SaaS-based content discovery platform designed to deliver fast, intelligent, and personalized search experiences across websites, portals, and applications.

Originally built on Reflektion technology (acquired by Sitecore in 2021), it enables teams to:

• Crawl content from websites, APIs, and structured feeds
• Organize and enrich content with metadata
• Deliver search results using built-in AI-powered relevance models
• Analyze user behavior to fine-tune relevance

Unlike Solr or Lucene tied to Sitecore XP/XM, Sitecore Search is a headless, standalone solution that works well in composable architectures.

Key Features of Sitecore Search

• Multi-source Crawling — HTML, JSON, XML, feeds, and APIs
• AI-Powered Ranking & Personalization — Learns from user behavior
• Predictive Search — Autocomplete, instant suggestions, popular queries
• Search Analytics — Track KPIs, CTRs, search terms, zero-result queries
• Composable Architecture — Easy integration with headless apps
• Search Widgets & SDKs — Drop-in UI or custom-built via APIs

Where Does Sitecore Search Fit?

Sitecore Search is part of Sitecore’s Composable DXP strategy. It integrates well with:

It’s ideal for public-facing content and not bound to Sitecore CMS, along with its federated search capabilities allowing multiple sources for search results.

How It Works

1. Crawler indexes web content or API responses
2. CEC (Customer Engagement Console) maps fields, sets up entities and filters
3. Search Service uses AI + behavior data to deliver ranked results
4. SDKs / REST API help render results on your frontend
5. Analytics Engine feeds back data for continuous optimization

Why Choose Sitecore Search?

Fast setup — No infrastructure needed, configure via CEC
Real-time tuning — Adjust results based on CTR, bounce rate, etc.
Improves discoverability — Especially across large or multisite ecosystems
Fully composable — Integrates with any frontend/backend system
Delivers results — More relevant search leads to better user engagement

Limitations and Considerations:

• Requires public-facing URLs (no internal/intranet content unless API-accessible)
• Schema design (attributes, filters, facets) should be planned up front

Summary

Sitecore Search is a powerful, composable SaaS solution for delivering fast, intelligent, personalized search on public-facing digital channels. It’s ideal for enterprises seeking SaaS scalability without re-engineering complex infrastructure.

Coming Up in This Blog Series

1. Getting Started — Setup, CEC walkthrough, creating attributes, configuring features
2. Crawlers & Sources — Detailed configuration of crawler/API sources and content Collection
3. Widgets— configurable components used to display search results
4. Analytics & Personalization — Event tracking and AI-driven tuning
5. Relevance Optimization — Managing boosts, synonyms, and synonyms