Skip to content

[BUG] Polling BlobTrigger stops detecting blobs when Storage Analytics logs are unavailable #61660

Description

@JoostVM

Library name and version

Microsoft.Azure.WebJobs.Extensions.Storage.Blobs 5.3.8, Azure.Storage.Blobs 12.26.0

Describe the bug

After upgrading to Microsoft.Azure.WebJobs.Extensions.Storage.Blobs 5.3.8, our classic (polling) BlobTrigger stopped detecting newly uploaded blobs when classic Storage Analytics logging (the $logs container) became unavailable due to an unrelated outage.

The library's blob polling strategy is documented as a hybrid between inspecting logs and running periodic container scans. The availability of classic storage analytics logs is described as "best effort". I would therefore expect the periodic container scan to continue detecting blobs even when storage analytics is unavailable and $logs contains no recent entries.

Instead, uploaded blobs were not picked up by our BlobTrigger. It did pick up blobs uploaded before its first scanning cycle. In effect, restarting the web job would result in it processing blobs uploaded before the restart, after which the trigger would stop working again.

Versions 5.3.7 and earlier do detect new blobs under these circumstances, although they miss some blobs when logs are not available - this is expected as it was exactly this behavior that 5.3.8 attempted to fix (#51030).

Suspected root cause

I suspect the container scan is ineffective because of how its terminal continuation token is evaluated.

Azure Blob Storage returns an empty terminal marker as <NextMarker /> . This means (with Azure.Storage.Blobs 12.26.0) the final Page<T>.ContinuationToken is observed as string.Empty, not null.

Upon starting a new scanning cycle, ScanBlobScanLogHybridPollingStrategy evaluates the token received at the end of the previous cycle against null.

            // if starting the cycle, reset the sweep time and set start time
            if (continuationToken == null)
            {
                containerScanInfo.PollingStartTime = DateTimeOffset.UtcNow;
                containerScanInfo.CurrentSweepCycleLatestModified = DateTimeOffset.MinValue;
            }

As a result, I suspect that this code is only reached on the first scan cycle, but not on subsequent cycles. This means PollingStartTime is not updated, and blobs uploaded after PollingStartTime remain undetected.

Expected behavior

When classic Storage Analytics logs are unavailable, delayed, or incomplete:

  1. The polling BlobTrigger should continue performing periodic container scans.
  2. A blob uploaded to the monitored container should eventually be detected by the scan.
  3. The function should execute, potentially with greater latency than when $logs is available.

Actual behavior

When the storage account is not producing classic Storage Analytics logs in the $logs container:

  1. Upon starting a web job using version 5.3.8, the BlobTrigger detects blobs that were uploaded before it starts running.
  2. A new blob uploaded to the container is not detected and the BlobTrigger is not invoked.

I believe this behavior is typically masked by the availability of storage analytics logs. The absence of logs exposes the fact that the container-scan portion is not functioning as intended.

Reproduction Steps

  1. Run a WebJob with a polling BlobTrigger using the container root as source [BlobTrigger("container/{name}")]
  2. Disable classic Storage Analytics on the targeted Storage Account
  3. Upload a new blob to the container
  4. Wait long enough for the periodic container scan to run
  5. Observe that the blob is not picked up and the triggered function is not invoked

Environment

  • Azure Storage standard GPv2 account
  • Polling BlobTrigger
  • Microsoft.Azure.WebJobs.Extensions.Storage.Blobs 5.3.8
  • Azure.Storage.Blobs 12.26.0
  • .NET 8 WebJob

Metadata

Metadata

Assignees

No one assigned

    Labels

    ClientThis issue points to a problem in the data-plane of the library.Service AttentionWorkflow: This issue is responsible by Azure service team.StorageStorage Service (Queues, Blobs, Files)customer-reportedIssues that are reported by GitHub users external to the Azure organization.needs-team-attentionWorkflow: This issue needs attention from Azure service team or SDK teamquestionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions