Library name and version
Microsoft.Azure.WebJobs.Extensions.Storage.Blobs 5.3.8, Azure.Storage.Blobs 12.26.0
Describe the bug
After upgrading to Microsoft.Azure.WebJobs.Extensions.Storage.Blobs 5.3.8, our classic (polling) BlobTrigger stopped detecting newly uploaded blobs when classic Storage Analytics logging (the $logs container) became unavailable due to an unrelated outage.
The library's blob polling strategy is documented as a hybrid between inspecting logs and running periodic container scans. The availability of classic storage analytics logs is described as "best effort". I would therefore expect the periodic container scan to continue detecting blobs even when storage analytics is unavailable and $logs contains no recent entries.
Instead, uploaded blobs were not picked up by our BlobTrigger. It did pick up blobs uploaded before its first scanning cycle. In effect, restarting the web job would result in it processing blobs uploaded before the restart, after which the trigger would stop working again.
Versions 5.3.7 and earlier do detect new blobs under these circumstances, although they miss some blobs when logs are not available - this is expected as it was exactly this behavior that 5.3.8 attempted to fix (#51030).
Suspected root cause
I suspect the container scan is ineffective because of how its terminal continuation token is evaluated.
Azure Blob Storage returns an empty terminal marker as <NextMarker /> . This means (with Azure.Storage.Blobs 12.26.0) the final Page<T>.ContinuationToken is observed as string.Empty, not null.
Upon starting a new scanning cycle, ScanBlobScanLogHybridPollingStrategy evaluates the token received at the end of the previous cycle against null.
// if starting the cycle, reset the sweep time and set start time
if (continuationToken == null)
{
containerScanInfo.PollingStartTime = DateTimeOffset.UtcNow;
containerScanInfo.CurrentSweepCycleLatestModified = DateTimeOffset.MinValue;
}
As a result, I suspect that this code is only reached on the first scan cycle, but not on subsequent cycles. This means PollingStartTime is not updated, and blobs uploaded after PollingStartTime remain undetected.
Expected behavior
When classic Storage Analytics logs are unavailable, delayed, or incomplete:
- The polling BlobTrigger should continue performing periodic container scans.
- A blob uploaded to the monitored container should eventually be detected by the scan.
- The function should execute, potentially with greater latency than when
$logs is available.
Actual behavior
When the storage account is not producing classic Storage Analytics logs in the $logs container:
- Upon starting a web job using version 5.3.8, the BlobTrigger detects blobs that were uploaded before it starts running.
- A new blob uploaded to the container is not detected and the BlobTrigger is not invoked.
I believe this behavior is typically masked by the availability of storage analytics logs. The absence of logs exposes the fact that the container-scan portion is not functioning as intended.
Reproduction Steps
- Run a WebJob with a polling BlobTrigger using the container root as source
[BlobTrigger("container/{name}")]
- Disable classic Storage Analytics on the targeted Storage Account
- Upload a new blob to the container
- Wait long enough for the periodic container scan to run
- Observe that the blob is not picked up and the triggered function is not invoked
Environment
- Azure Storage standard GPv2 account
- Polling BlobTrigger
Microsoft.Azure.WebJobs.Extensions.Storage.Blobs 5.3.8
Azure.Storage.Blobs 12.26.0
- .NET 8 WebJob
Library name and version
Microsoft.Azure.WebJobs.Extensions.Storage.Blobs 5.3.8, Azure.Storage.Blobs 12.26.0
Describe the bug
After upgrading to
Microsoft.Azure.WebJobs.Extensions.Storage.Blobs5.3.8, our classic (polling) BlobTrigger stopped detecting newly uploaded blobs when classic Storage Analytics logging (the$logscontainer) became unavailable due to an unrelated outage.The library's blob polling strategy is documented as a hybrid between inspecting logs and running periodic container scans. The availability of classic storage analytics logs is described as "best effort". I would therefore expect the periodic container scan to continue detecting blobs even when storage analytics is unavailable and
$logscontains no recent entries.Instead, uploaded blobs were not picked up by our BlobTrigger. It did pick up blobs uploaded before its first scanning cycle. In effect, restarting the web job would result in it processing blobs uploaded before the restart, after which the trigger would stop working again.
Versions 5.3.7 and earlier do detect new blobs under these circumstances, although they miss some blobs when logs are not available - this is expected as it was exactly this behavior that 5.3.8 attempted to fix (#51030).
Suspected root cause
I suspect the container scan is ineffective because of how its terminal continuation token is evaluated.
Azure Blob Storage returns an empty terminal marker as
<NextMarker />. This means (withAzure.Storage.Blobs12.26.0) the finalPage<T>.ContinuationTokenis observed asstring.Empty, notnull.Upon starting a new scanning cycle,
ScanBlobScanLogHybridPollingStrategyevaluates the token received at the end of the previous cycle againstnull.As a result, I suspect that this code is only reached on the first scan cycle, but not on subsequent cycles. This means PollingStartTime is not updated, and blobs uploaded after PollingStartTime remain undetected.
Expected behavior
When classic Storage Analytics logs are unavailable, delayed, or incomplete:
$logsis available.Actual behavior
When the storage account is not producing classic Storage Analytics logs in the
$logscontainer:I believe this behavior is typically masked by the availability of storage analytics logs. The absence of logs exposes the fact that the container-scan portion is not functioning as intended.
Reproduction Steps
[BlobTrigger("container/{name}")]Environment
Microsoft.Azure.WebJobs.Extensions.Storage.Blobs5.3.8Azure.Storage.Blobs12.26.0