[Bug report] InputImage.fromMediaImage not applying rotation

[eebirke]

Describe the bug
InputImage.fromMediaImage and InputImage.fromBitmap do not seem to be applying parameter rotationDegrees properly, at least when used with text-recognition.

I'm attempting to fix MLKit results when used with react-native-vision-camera, and discovered that all rotationDegrees values supplied as input result in the exact same detection outcomes as not rotating the image at all (180 degrees rotation from upright sensor returns text lines in the opposite order). The data is YUV_420_888 directly from Camera2/CameraX as Image.

Converting the YUV to Bitmap manually and using InputImage.fromBitmap returned the same results.

The only way I could circumvent this was by rotating the bitmap before creating the InputImage, or setting automatic rotation of the buffers in CameraX.

My original issue in vision-camera mrousavy/react-native-vision-camera#3490

To Reproduce
Take Image from CameraX or Camera 2 when rotated to 180 degrees from upright sensor, process with InputImage, and process with TextRecognizer. Check the order of the lines in returned text.

Getting the sample app up and running and modifying all relevant parts to reproduce this is a huge task for me, but here's the callback logic I have tested with vision-camera

  private var recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)

  override fun callback(frame: Frame, arguments: Map<String, Any>?): HashMap<String, Any>? {
        val data = WritableNativeMap()
        val mediaImage: Image = frame.image
        val image =
            InputImage.fromMediaImage(mediaImage, 180) // Hard coded for testing purposes, no values worked as expected
        val task: Task<Text> = recognizer.process(image)
        try {
            val text: Text = Tasks.await(task)
            if (text.text.isEmpty()) {
                return WritableNativeMap().toHashMap()
            }
            data.putString("resultText", text.text) // This text comes in reversed order when sensor is at 180 deg
            data.putArray("blocks", getBlocks(text.textBlocks))
            return data.toHashMap()
        } catch (e: Exception) {
            e.printStackTrace()
            return null
        }
    }

Expected behavior
Expect resulting MediaImage to be rotated and result in expected Text Recognition output ordering.

SDK Info:

• com.google.android.gms:play-services-mlkit-text-recognition:19.0.1
• com.google.android.gms:play-services-mlkit-text-recognition-chinese:16.0.1 (+ others at same version)
• com.google.mlkit:translate:17.0.3
  Also tested with latest non-play services variants.

Smartphone:

• Device/Simulator: Pixel 9 Pro, Asus Zenfone 8
• Device/Simulator OS: Android API 33, Android API 35

Development Environment:
(For Android issue feel free to skip this section)

• IDE Eversion: Android Studio Meerkat | 2024.3.1 Patch 1
• Laptop/Desktop: MacBook Pro M3 Max
• Laptop/Desktop OS/version: MacOS 15.4