Skip to content

Conversation

@eolecvk
Copy link
Contributor

@eolecvk eolecvk commented Oct 12, 2022

  • Fixed conditional logic when using autocast in benchmark code.
  • Updated diffusers==0.3.0 to diffusers==0.4.2
  • @chuanli11 : Any reason to not include torch and torchvision in the requirements?

Removing autocast does seem to reduce the latency and memory in a big way!
Here is what the new benchmark results for autocast v fp16 look like.

device precision autocast runtime n_samples latency memory
NVIDIA RTX A6000 single False pytorch 1 7.37 7.73
NVIDIA RTX A6000 half True pytorch 1 4.39 4.52
NVIDIA RTX A6000 half False pytorch 1 3.34 3.38
NVIDIA RTX A6000 single False pytorch 2 14.0 9.57
NVIDIA RTX A6000 half True pytorch 2 8.29 8.74
NVIDIA RTX A6000 half False pytorch 2 6.17 4.92
NVIDIA RTX A6000 single False pytorch 4 27.25 18.4
NVIDIA RTX A6000 half True pytorch 4 15.73 11.4
NVIDIA RTX A6000 half False pytorch 4 11.59 8.82

Also added benchmark config instructions in benchmark.md

@eolecvk eolecvk requested a review from chuanli11 October 12, 2022 19:02
@chuanli11
Copy link
Contributor

@eolecvk seems torch and torchvision are included in the setup.py.

I don't know if @justinpinkney has particular reason for doing it that way.

@eolecvk
Copy link
Contributor Author

eolecvk commented Oct 12, 2022

@eolecvk seems torch and torchvision are included in the setup.py.

I don't know if @justinpinkney has particular reason for doing it that way.

It's because the original repo was setup that way:
https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/setup.py

...but even in their case, they install pytorch and torchvision in conda's environment.yaml:
https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/environment.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants