Enable tests by adding separate torchcodec-tests output [Debugging] #5

traversaro · 2025-04-24T17:16:55Z

As discussed in conda-forge/staged-recipes#29447 (comment) . The structure of using a separate (empty) torchcodec-tests output that only contains the (heavyweight) files used for tests is inspired from the torchvision-feedstock.

I also removed the manual pin of libtorch and pytorch to 2.6.* as now the global pinning are using 2.6, see conda-forge/conda-forge-pinning-feedstock#7208 .

Checklist

Used a personal fork of the feedstock to propose changes
Bumped the build number (if the version is unchanged)
Reset the build number to 0 (if the version changed)
Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
Ensured the license file is being packaged.

traversaro · 2025-04-24T17:17:09Z

@conda-forge-admin, please rerender

conda-forge-admin · 2025-04-24T17:18:29Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found some lint.

Here's what I've got...

For recipe/recipe.yaml:

❌ There are too few lines. There should be one empty line at the end of the file.

_{This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/14647613176. Examine the logs at this URL for more detail.}

…nda-forge-pinning 2025.04.23.13.34.55

conda-forge-admin · 2025-04-24T17:20:34Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.

traversaro · 2025-04-24T17:22:53Z

@conda-forge-admin please rerender

…nda-forge-pinning 2025.04.23.13.34.55

traversaro · 2025-04-24T17:59:31Z

This PR requires conda-forge/admin-requests#1501 to be merged for the CI to be green.

traversaro · 2025-04-25T12:52:25Z

Test on linux-aarch64 are failing (just the top of the output, the failure are much longer):

 │ + pytest test
 │ ============================= test session starts ==============================
 │ platform linux -- Python 3.13.3, pytest-8.3.5, pluggy-1.5.0
 │ rootdir: $SRC_DIR
 │ collected 308 items
 │ test/decoders/test_decoders.py .................FFssFFssFFsFs.s.sFsFsFFs [ 13%]
 │ Fs.s.s.sFsFs.s.sFsFsFsF.s.sFsFs.s.sFsFsFsFsFsFs..............sssssssssss [ 36%]
 │ sss..............ssssssssssssss.FFFsssFFFsss.s.s.sFs.                    [ 53%]
 │ test/decoders/test_metadata.py ........                                  [ 56%]
 │ test/decoders/test_ops.py FsFsFsFsFsFs.s.s.sFsFs.sFsFsFsFs.....FF.....FF [ 71%]
 │ FF...............s.......                                                [ 79%]
 │ test/samplers/test_policy.py .....                                       [ 81%]
 │ test/samplers/test_samplers.py ........FFFF............................. [ 94%]
 │ .......                                                                  [ 96%]
 │ test/samplers/test_video_clip_sampler.py ....                            [ 98%]
 │ test/test_frame_dataclasses.py .....                                     [ 99%]
 │ test/test_version.py .                                                   [100%]
 │ =================================== FAILURES ===================================
 │ ________________ TestVideoDecoder.test_getitem_int[exact-cpu-1] ________________
 │ self = <test.decoders.test_decoders.TestVideoDecoder object at 0x40005d6049d0>
 │ num_ffmpeg_threads = 1, device = 'cpu', seek_mode = 'exact'
 │     @pytest.mark.parametrize("num_ffmpeg_threads", (1, 4))
 │     @pytest.mark.parametrize("device", cpu_and_cuda())
 │     @pytest.mark.parametrize("seek_mode", ("exact", "approximate"))
 │     def test_getitem_int(self, num_ffmpeg_threads, device, seek_mode):
 │         decoder = VideoDecoder(
 │             NASA_VIDEO.path,
 │             num_ffmpeg_threads=num_ffmpeg_threads,
 │             device=device,
 │             seek_mode=seek_mode,
 │         )
 │     
 │         ref_frame0 = NASA_VIDEO.get_frame_data_by_index(0).to(device)
 │         ref_frame1 = NASA_VIDEO.get_frame_data_by_index(1).to(device)
 │         ref_frame180 = NASA_VIDEO.get_frame_data_by_index(180).to(device)
 │         ref_frame_last = NASA_VIDEO.get_frame_data_by_index(289).to(device)
 │     
 │ >       assert_frames_equal(ref_frame0, decoder[0])
 │ test/decoders/test_decoders.py:105:

traversaro · 2025-04-25T12:54:27Z

I am not sure if the failure is due to the emulation or due to linux-aarch64, I probably need to check on a physical linux-aarch64 machine.

Copilot

Pull Request Overview

This PR updates the CI configurations to enable tests with a separate torchcodec-tests output and removes redundant version pinning for libtorch and pytorch by aligning them with the global pin (set to 2.6). Key changes include:

Adding "libtorch" pin entries with version '2.6' across multiple YAML config files.
Adding "pytorch" pin entries with version '2.6' across the same configuration files.
Updating the Azure Pipelines matrix to reference the new cuda_compiler-based configuration names.

Reviewed Changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
.ci_support/linux_aarch64_cuda_compiler_versionNonepython3.13.____cp313.yaml	Added libtorch and pytorch version pins
.ci_support/linux_aarch64_cuda_compiler_versionNonepython3.12.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_aarch64_cuda_compiler_versionNonepython3.11.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_aarch64_cuda_compiler_versionNonepython3.10.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_aarch64_cuda_compiler_version12.6python3.9.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_aarch64_cuda_compiler_version12.6python3.13.____cp313.yaml	Added libtorch and pytorch version pins
.ci_support/linux_aarch64_cuda_compiler_version12.6python3.12.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_aarch64_cuda_compiler_version12.6python3.11.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_aarch64_cuda_compiler_version12.6python3.10.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_versionNonepython3.9.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_versionNonepython3.13.____cp313.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_versionNonepython3.12.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_versionNonepython3.11.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_versionNonepython3.10.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_version12.6python3.9.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_version12.6python3.13.____cp313.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_version12.6python3.12.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_version12.6python3.11.____cpython.yaml	Added libtorch and pytorch version pins
.ci_support/linux_64_cuda_compiler_version12.6python3.10.____cpython.yaml	Added libtorch and pytorch version pins
.azure-pipelines/azure-pipelines-linux.yml	Updated matrix configuration keys to reference the new cuda_compiler-based configurations

traversaro · 2025-04-25T13:00:48Z

As the linux-aarch64 builds are already available (and with similar failures), I do not think it make sense to block this PR on linux-aarch64 passing all tests, we can just merge by skipping tests on linux-aarch64, so that at least all the other architectures work.

traversaro · 2025-04-25T13:01:15Z

As the linux-aarch64 builds are already available (and with similar failures), I do not think it make sense to block this PR on linux-aarch64 passing all tests, we can just merge by skipping tests on linux-aarch64, so that at least all the other architectures work.

Actually an attempt we can do is to try emulated builds instead of cross-compilation.

…tion

traversaro · 2025-04-25T13:02:19Z

@conda-forge-admin please rerender

…nda-forge-pinning 2025.04.25.09.56.22

traversaro · 2025-04-25T13:09:40Z

@conda-forge-admin please rerender

…nda-forge-pinning 2025.04.25.09.56.22

traversaro · 2025-04-26T09:41:30Z

The test failures are there also switching from cross-compilation to emulation, so they are probably not related to cross-compilation. They may be related to the emulation of tests, we should try to run them without emulation, and see if they fail also there. The same failures are there also on 0.3.0, see: #6 .

traversaro · 2025-04-27T14:08:39Z

In 305374a I switched back to cross-compilation.

traversaro · 2025-04-27T14:08:49Z

@conda-forge-admin please rerender

…nda-forge-pinning 2025.04.25.23.47.09

traversaro · 2025-04-27T15:04:52Z

As this PR turned out to be more a debugging sandbox, I opened #7 for the PR for actually adding tests.

conda-forge-admin · 2025-04-27T15:30:03Z

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict. Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

conda-forge-admin · 2025-04-27T15:39:11Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/recipe.yaml) and found it was in an excellent condition.

traversaro · 2025-04-28T07:13:47Z

The test failures have been reproduced in https://github.com/traversaro/torchcodec-test-debug and reported in pytorch/torchcodec#569 (comment), so we can close this.

Enable tests by adding separate torchcodec-tests output

9817555

traversaro and others added 3 commits April 24, 2025 19:19

Add missing new line

893f337

MNT: Re-rendered with conda-build 25.4.2, conda-smithy 3.47.2, and co…

94df0e4

…nda-forge-pinning 2025.04.23.13.34.55

Remove workaround of explicit libtorch and pytorch pin

babf251

MNT: Re-rendered with conda-build 25.4.2, conda-smithy 3.47.2, and co…

d38f864

…nda-forge-pinning 2025.04.23.13.34.55

traversaro mentioned this pull request Apr 24, 2025

Add torchcodec-tests output in the torchcodec feedstock conda-forge/admin-requests#1501

Merged

3 tasks

traversaro closed this Apr 24, 2025

traversaro reopened this Apr 24, 2025

traversaro requested a review from Copilot April 25, 2025 12:55

Copilot AI reviewed Apr 25, 2025

View reviewed changes

Switch linux_aarch64 builds to use emulation instead of cross-compila…

9dccacc

…tion

conda-forge-webservices[bot] and others added 2 commits April 25, 2025 13:04

MNT: Re-rendered with conda-build 25.4.2, conda-smithy 3.47.2, and co…

587b0b4

…nda-forge-pinning 2025.04.25.09.56.22

Switch linux_aarch64 to use build with emulation

c627bbc

MNT: Re-rendered with conda-build 25.4.2, conda-smithy 3.47.2, and co…

753f653

…nda-forge-pinning 2025.04.25.09.56.22

traversaro mentioned this pull request Apr 26, 2025

Enable tests by adding separate torchcodec-tests output and update torchcodec to 0.3.0 #6

Closed

5 tasks

Update conda-forge.yml

305374a

conda-forge-webservices[bot] and others added 2 commits April 27, 2025 14:10

MNT: Re-rendered with conda-build 25.4.2, conda-smithy 3.47.2, and co…

e4085fe

…nda-forge-pinning 2025.04.25.23.47.09

Add C++ tests

93465aa

traversaro changed the title ~~Enable tests by adding separate torchcodec-tests output~~ Enable tests by adding separate torchcodec-tests output [Debugging] Apr 27, 2025

traversaro added 3 commits April 27, 2025 16:52

Create increase_timeout.patch

9a14333

Update recipe.yaml

c9e3f1d

Update build.sh

cdd8107

traversaro mentioned this pull request Apr 27, 2025

Enable tests by adding separate torchcodec-tests output and disable linux-aarch64 builds #7

Merged

5 tasks

traversaro added 3 commits April 27, 2025 17:07

Update increase_timeout.patch

d29c136

Update increase_timeout.patch

84e579e

Update build.sh

b2ddaee

Merge branch 'main' into addtests

50fcaaf

traversaro closed this Apr 28, 2025

Uh oh!

Enable tests by adding separate torchcodec-tests output [Debugging] #5

Enable tests by adding separate torchcodec-tests output [Debugging] #5

Uh oh!

Conversation

traversaro commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

traversaro commented Apr 24, 2025

Uh oh!

conda-forge-admin commented Apr 24, 2025

Uh oh!

conda-forge-admin commented Apr 24, 2025

Uh oh!

traversaro commented Apr 24, 2025

Uh oh!

traversaro commented Apr 24, 2025

Uh oh!

traversaro commented Apr 25, 2025

Uh oh!

traversaro commented Apr 25, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

traversaro commented Apr 25, 2025

Uh oh!

traversaro commented Apr 25, 2025

Uh oh!

traversaro commented Apr 25, 2025

Uh oh!

traversaro commented Apr 25, 2025

Uh oh!

traversaro commented Apr 26, 2025

Uh oh!

traversaro commented Apr 27, 2025

Uh oh!

traversaro commented Apr 27, 2025

Uh oh!

traversaro commented Apr 27, 2025

Uh oh!

conda-forge-admin commented Apr 27, 2025

Uh oh!

conda-forge-admin commented Apr 27, 2025

Uh oh!

traversaro commented Apr 28, 2025

Uh oh!

Uh oh!

traversaro commented Apr 24, 2025 •

edited

Loading