[Attachment Support] Support dragging attachment elements out as files on iOS

https://bugs.webkit.org/show_bug.cgi?id=181199
<rdar://problem/36299316>

Reviewed by Tim Horton, Andy Estes and Joseph Pecoraro.

Source/WebCore:

Adds support for dragging "files" (i.e. creating item providers with preferred attachment presentation styles)
from attachment elements on iOS for Mail. See below for more detail.

Tests:  WKAttachmentTestsIOS.DragAttachmentInsertedAsData
        WKAttachmentTestsIOS.DragAttachmentInsertedAsFile

* page/DragController.cpp:
(WebCore::DragController::platformContentTypeForBlobType const):
(WebCore::DragController::dragAttachmentElement):
* page/DragController.h:
* page/mac/DragControllerMac.mm:
(WebCore::DragController::platformContentTypeForBlobType const):

Add a private method to convert the type of a promised blob to a platform type. For Cocoa platforms, this
converts the blob type (either a UTI or a MIME type) to a UTI for the platform to consume.

* platform/ios/WebItemProviderPasteboard.h:
* platform/ios/WebItemProviderPasteboard.mm:

Refactor WebItemProviderRegistrationInfo. WebItemProviderRegistrationInfo currently encapsulates a single item
provider registration call, and contains either a type identifier and data buffer, or an NSItemProviderWriting-
conformant object. To register an item provider using a WebItemProviderRegistrationInfo, the item provider
pasteboard currently checks to see whether the info contains an object or a type and data.

This patch removes WebItemProviderRegistrationInfo and replaces it with WebItemProviderDataRegistrar. Objects
that implement this protocol know how to take an NSItemProvider and register data to it. So far, there are
three implementations below.

(-[WebItemProviderDataRegistrar initWithData:type:]):
(-[WebItemProviderDataRegistrar typeIdentifier]):
(-[WebItemProviderDataRegistrar data]):
(-[WebItemProviderDataRegistrar typeIdentifierForClient]):
(-[WebItemProviderDataRegistrar dataForClient]):
(-[WebItemProviderDataRegistrar registerItemProvider:]):
(-[WebItemProviderDataRegistrar description]):

A data registrar takes a UTI and data buffer, and registers the UTI to the data. This replaces a
WebItemProviderRegistrationInfo with both a type and data, but no representing object.

(-[WebItemProviderWritableObjectRegistrar initWithObject:]):
(-[WebItemProviderWritableObjectRegistrar representingObjectForClient]):
(-[WebItemProviderWritableObjectRegistrar registerItemProvider:]):
(-[WebItemProviderWritableObjectRegistrar description]):

The writable object registrar writes an NSItemProviderWriting-conformant object to an item provider. This
replaces a WebItemProviderRegistrationInfo with only a representing object.

(-[WebItemProviderPromisedFileRegistrar initWithType:callback:]):
(-[WebItemProviderPromisedFileRegistrar registerItemProvider:]):
(-[WebItemProviderPromisedFileRegistrar description]):
(-[WebItemProviderRegistrationInfoList addData:forType:]):
(-[WebItemProviderRegistrationInfoList addRepresentingObject:]):
(-[WebItemProviderRegistrationInfoList addPromisedType:fileCallback:]):

Helper methods to add new registrars to a registration info list.

(-[WebItemProviderRegistrationInfoList itemAtIndex:]):
(-[WebItemProviderRegistrationInfoList enumerateItems:]):
(-[WebItemProviderRegistrationInfoList itemProvider]):
(-[WebItemProviderRegistrationInfoList description]):
(-[WebItemProviderRegistrationInfo initWithRepresentingObject:typeIdentifier:data:]): Deleted.
(-[WebItemProviderRegistrationInfo representingObject]): Deleted.
(-[WebItemProviderRegistrationInfo typeIdentifier]): Deleted.

Source/WebKit:

Implement support for registering and beginning a drag with promised blob info. See below for more detail.

* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKDragSessionContext addTemporaryDirectory:]):
(-[WKDragSessionContext cleanUpTemporaryDirectories]):

Introduce WKDragSessionContext, which represents the localContext of a UIDragSession initiated in WebKit. The
blob promise dragging codepath uses this to register temporary directories when saving blob data to a location
on disk; when all data transfers are finished, or if the drag interaction is being reset, we then use
-cleanUpTemporaryDirectories to remove each temporary directory.

(existingLocalDragSessionContext):
(ensureLocalDragSessionContext):

Helper methods to set the UIDragSession's localContext to a WKDragSessionContext and query for any existing
context.

(-[WKContentView cleanupInteraction]):

Before the content view's UIDragInteraction goes away, clean up any temporary directories added to the
UIDragSession.

(-[WKContentView _prepareToDragPromisedBlob:]):

When dragging with a promised blob, register a new item provider on the pasteboard representing the blob data,
along with any additional metadata associated with the blob. For the promise callback, call out to the network
process to write the blob data to a temporary path; when done, call the NSItemProvider's completion handler with
the temporary blob data location.

(-[WKContentView _itemsForBeginningOrAddingToSessionWithRegistrationList:stagedDragSource:]):
(-[WKContentView dragInteraction:sessionDidTransferItems:]):

Use this delegate hook as an opportunity to remove any temporary directories created when promised blob data is
requested upon drop. Since we know the drag session that has finished transferring data, we simply ask its local
context (a WKDragSessionContext) to remove any temporary filepaths it has created.

Tools:

Add support in the drag and drop simulator for testing blob-backed attachment element dragging, and also add new
attachment API tests.

* TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm:
(-[NSItemProvider expectType:withData:]):
(TestWebKitAPI::TEST):

Add two new WKAttachmentTests to exercise dragging data- and file-backed blobs via attachment elements. These
tests first insert attachments via drop or WKWebView SPI, and then drag these attachments out and use the
-expectType:withData: helper to inspect the item providers created from the drag source.

* TestWebKitAPI/Tests/ios/DataInteractionTests.mm:
(TestWebKitAPI::TEST):
* TestWebKitAPI/ios/DataInteractionSimulator.h:
* TestWebKitAPI/ios/DataInteractionSimulator.mm:
(-[MockDragSession localContext]):
(-[MockDragSession setLocalContext:]):
(-[DataInteractionSimulator _resetSimulatedState]):
(-[DataInteractionSimulator simulateAllTouchesCanceled:]):
(-[DataInteractionSimulator _concludeDataInteractionAndPerformOperationIfNecessary]):
(-[DataInteractionSimulator _advanceProgress]):
(-[DataInteractionSimulator endDataTransfer]):

Make some tweaks to the iOS drag and drop simulator. In particular, this patch (1) adds a new hook to tell
WebKit that data transfers have been completed, (2) fixes incorrect drop proposal handling when returning
UIDropOperationForbidden by replacing _shouldPerformOperation with a UIDropProposal, and (3) teach the
MockDragSession to hold on to a localContext.


git-svn-id: http://svn.webkit.org/repository/webkit/trunk@226779 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Author: [email protected]

Source/WebCore/ChangeLog

@@ -1,3 +1,75 @@
+2018-01-11  Wenson Hsieh  <[email protected]>
+
+        [Attachment Support] Support dragging attachment elements out as files on iOS
+        https://bugs.webkit.org/show_bug.cgi?id=181199
+        <rdar://problem/36299316>
+
+        Reviewed by Tim Horton, Andy Estes and Joseph Pecoraro.
+
+        Adds support for dragging "files" (i.e. creating item providers with preferred attachment presentation styles)
+        from attachment elements on iOS for Mail. See below for more detail.
+
+        Tests:  WKAttachmentTestsIOS.DragAttachmentInsertedAsData
+                WKAttachmentTestsIOS.DragAttachmentInsertedAsFile
+
+        * page/DragController.cpp:
+        (WebCore::DragController::platformContentTypeForBlobType const):
+        (WebCore::DragController::dragAttachmentElement):
+        * page/DragController.h:
+        * page/mac/DragControllerMac.mm:
+        (WebCore::DragController::platformContentTypeForBlobType const):
+
+        Add a private method to convert the type of a promised blob to a platform type. For Cocoa platforms, this
+        converts the blob type (either a UTI or a MIME type) to a UTI for the platform to consume.
+
+        * platform/ios/WebItemProviderPasteboard.h:
+        * platform/ios/WebItemProviderPasteboard.mm:
+
+        Refactor WebItemProviderRegistrationInfo. WebItemProviderRegistrationInfo currently encapsulates a single item
+        provider registration call, and contains either a type identifier and data buffer, or an NSItemProviderWriting-
+        conformant object. To register an item provider using a WebItemProviderRegistrationInfo, the item provider
+        pasteboard currently checks to see whether the info contains an object or a type and data.
+
+        This patch removes WebItemProviderRegistrationInfo and replaces it with WebItemProviderDataRegistrar. Objects
+        that implement this protocol know how to take an NSItemProvider and register data to it. So far, there are
+        three implementations below.
+
+        (-[WebItemProviderDataRegistrar initWithData:type:]):
+        (-[WebItemProviderDataRegistrar typeIdentifier]):
+        (-[WebItemProviderDataRegistrar data]):
+        (-[WebItemProviderDataRegistrar typeIdentifierForClient]):
+        (-[WebItemProviderDataRegistrar dataForClient]):
+        (-[WebItemProviderDataRegistrar registerItemProvider:]):
+        (-[WebItemProviderDataRegistrar description]):
+
+        A data registrar takes a UTI and data buffer, and registers the UTI to the data. This replaces a
+        WebItemProviderRegistrationInfo with both a type and data, but no representing object.
+
+        (-[WebItemProviderWritableObjectRegistrar initWithObject:]):
+        (-[WebItemProviderWritableObjectRegistrar representingObjectForClient]):
+        (-[WebItemProviderWritableObjectRegistrar registerItemProvider:]):
+        (-[WebItemProviderWritableObjectRegistrar description]):
+
+        The writable object registrar writes an NSItemProviderWriting-conformant object to an item provider. This
+        replaces a WebItemProviderRegistrationInfo with only a representing object.
+
+        (-[WebItemProviderPromisedFileRegistrar initWithType:callback:]):
+        (-[WebItemProviderPromisedFileRegistrar registerItemProvider:]):
+        (-[WebItemProviderPromisedFileRegistrar description]):
+        (-[WebItemProviderRegistrationInfoList addData:forType:]):
+        (-[WebItemProviderRegistrationInfoList addRepresentingObject:]):
+        (-[WebItemProviderRegistrationInfoList addPromisedType:fileCallback:]):
+
+        Helper methods to add new registrars to a registration info list.
+
+        (-[WebItemProviderRegistrationInfoList itemAtIndex:]):
+        (-[WebItemProviderRegistrationInfoList enumerateItems:]):
+        (-[WebItemProviderRegistrationInfoList itemProvider]):
+        (-[WebItemProviderRegistrationInfoList description]):
+        (-[WebItemProviderRegistrationInfo initWithRepresentingObject:typeIdentifier:data:]): Deleted.
+        (-[WebItemProviderRegistrationInfo representingObject]): Deleted.
+        (-[WebItemProviderRegistrationInfo typeIdentifier]): Deleted.
+
 2018-01-11  Michael Saboff  <[email protected]>
 
         Add a DOM gadget for Spectre testing

Source/WebCore/page/DragController.cpp

@@ -1274,6 +1274,15 @@ bool DragController::shouldUseCachedImageForDragImage(const Image& image) const
 #endif
 }
 
+#if !PLATFORM(COCOA)
+
+String DragController::platformContentTypeForBlobType(const String& type) const
+{
+    return type;
+}
+
+#endif
+
 #if ENABLE(ATTACHMENT_ELEMENT)
 
 bool DragController::dragAttachmentElement(Frame& frame, HTMLAttachmentElement& attachment)
@@ -1289,7 +1298,7 @@ bool DragController::dragAttachmentElement(Frame& frame, HTMLAttachmentElement&
 #endif
 
     auto& file = *attachment.file();
-    m_client.prepareToDragPromisedBlob({ file.url(), file.type(), file.name(), additionalTypes, additionalData });
+    m_client.prepareToDragPromisedBlob({ file.url(), platformContentTypeForBlobType(file.type()), file.name(), WTFMove(additionalTypes), WTFMove(additionalData) });
 
     return true;
 }

Source/WebCore/page/DragController.h

@@ -128,6 +128,8 @@ struct DragState;
 #endif
         }
 
+        String platformContentTypeForBlobType(const String& type) const;
+
         void cleanupAfterSystemDrag();
         void declareAndWriteDragImage(DataTransfer&, Element&, const URL&, const String& label);
 

Source/WebCore/page/mac/DragControllerMac.mm

@@ -46,6 +46,7 @@
 #import "PlatformStrategies.h"
 #import "Range.h"
 #import "RuntimeEnabledFeatures.h"
+#import "UTIUtilities.h"
 
 #if ENABLE(DATA_INTERACTION)
 #import <MobileCoreServices/MobileCoreServices.h>
@@ -91,6 +92,14 @@
     return maxDragImageSize;
 }
 
+String DragController::platformContentTypeForBlobType(const String& type) const
+{
+    auto utiType = UTIFromMIMEType(type);
+    if (!utiType.isEmpty())
+        return utiType;
+    return type;
+}
+
 void DragController::cleanupAfterSystemDrag()
 {
 #if PLATFORM(MAC)

Source/WebCore/platform/ios/WebItemProviderPasteboard.h

@@ -40,20 +40,23 @@ typedef NS_ENUM(NSInteger, WebPreferredPresentationStyle) {
 
 NS_ASSUME_NONNULL_BEGIN
 
-/*! A WebItemProviderRegistrationInfo represents a single call to register something to an item provider.
- @discussion Either the representing object exists and the type identifier and data are nil, or the
- representing object is nil and the type identifier and data exist. The former represents a call to
- register an entire UIItemProviderWriting-conformant object to the item provider, while the latter
- represents a call to register only a data representation for the given type identifier.
+/*! A WebItemProviderRegistrar encapsulates a single call to register something to an item provider.
+ @discussion Classes that implement this protocol each represent a different way of writing data to
+ an item provider. Some examples include setting a chunk of data corresponding to a type identifier,
+ or registering a NSItemProviderWriting-conformant object, or registering a type to a promised file
+ where the data has been written.
  */
-WEBCORE_EXPORT @interface WebItemProviderRegistrationInfo : NSObject
-
-@property (nonatomic, readonly, nullable, strong) id <UIItemProviderWriting> representingObject;
-@property (nonatomic, readonly, nullable, strong) NSString *typeIdentifier;
-@property (nonatomic, readonly, nullable, strong) NSData *data;
+@protocol WebItemProviderRegistrar <NSObject>
+- (void)registerItemProvider:(NSItemProvider *)itemProvider;
 
+@optional
+@property (nonatomic, readonly) id <NSItemProviderWriting> representingObjectForClient;
+@property (nonatomic, readonly) NSString *typeIdentifierForClient;
+@property (nonatomic, readonly) NSData *dataForClient;
 @end
 
+typedef void(^WebItemProviderFileCallback)(NSURL * _Nullable, NSError * _Nullable);
+
 /*! A WebItemProviderRegistrationInfoList represents a series of registration calls used to set up a
  single item provider.
  @discussion The order of items specified in the list (lowest indices first) is the order in which
@@ -65,6 +68,7 @@ WEBCORE_EXPORT @interface WebItemProviderRegistrationInfoList : NSObject
 
 - (void)addRepresentingObject:(id <UIItemProviderWriting>)object;
 - (void)addData:(NSData *)data forType:(NSString *)typeIdentifier;
+- (void)addPromisedType:(NSString *)typeIdentifier fileCallback:(void(^)(WebItemProviderFileCallback))callback;
 
 @property (nonatomic) CGSize preferredPresentationSize;
 @property (nonatomic, copy) NSString *suggestedName;
@@ -74,8 +78,8 @@ WEBCORE_EXPORT @interface WebItemProviderRegistrationInfoList : NSObject
 @property (nonatomic, copy) NSData *teamData;
 
 - (NSUInteger)numberOfItems;
-- (nullable WebItemProviderRegistrationInfo *)itemAtIndex:(NSUInteger)index;
-- (void)enumerateItems:(void(^)(WebItemProviderRegistrationInfo *item, NSUInteger index))block;
+- (nullable id <WebItemProviderRegistrar>)itemAtIndex:(NSUInteger)index;
+- (void)enumerateItems:(void(^)(id <WebItemProviderRegistrar> item, NSUInteger index))block;
 
 @end