<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by learn2train.org on Medium]]></title>
        <description><![CDATA[Stories by learn2train.org on Medium]]></description>
        <link>https://medium.com/@learn2train?source=rss-6d74d24f9c45------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*n40X9uD8O7nTvNNJ0VygEQ.png</url>
            <title>Stories by learn2train.org on Medium</title>
            <link>https://medium.com/@learn2train?source=rss-6d74d24f9c45------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 15:46:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@learn2train/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Fine-tuning Wan 2.1 With a Curated Dataset on the GPU Cloud (Step-by-Step Guide)]]></title>
            <link>https://learn2train.medium.com/fine-tuning-wan-2-1-with-a-curated-dataset-step-by-step-guide-a6f0b334ab79?source=rss-6d74d24f9c45------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6f0b334ab79</guid>
            <category><![CDATA[deepfake-videos]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[comfy-ui]]></category>
            <category><![CDATA[wan-2-1]]></category>
            <category><![CDATA[vi̇deo]]></category>
            <dc:creator><![CDATA[learn2train.org]]></dc:creator>
            <pubDate>Sat, 15 Mar 2025 13:05:01 GMT</pubDate>
            <atom:updated>2026-10-11T12:06:34.477Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*doS82-n5_BxaUEKiGAZ48w.gif" /><figcaption>Is Monica Vitti starring in an upcoming Netflix series?</figcaption></figure><p>You might not recognise the scene above, as it is from a new Netflix series to be released at the end of this year. The elegant lady holding the glass is none other than <a href="https://en.wikipedia.org/wiki/Monica_Vitti">Monica Vitti</a>, an Italian actor who starred in some of the most famous works of the renowned Italian film director <a href="https://www.britannica.com/biography/Michelangelo-Antonioni">Michelangelo Antonioni</a>, such as <a href="https://www.criterion.com/films/209-l-avventura"><em>L’Avventura</em></a>, <a href="https://www.imdb.com/title/tt0054130/"><em>La Notte</em></a> or <a href="https://www.amazon.com/LEclisse-Criterion-Collection-Alain-Delon/dp/B0007989Y8"><em>L’Eclisse</em></a>.</p><p>Now, if you know who Monica Vitti is, you already know that I’m totally b*llshitting here. There is no such Netflix series coming up. In fact, the scene above is a complete mirage…</p><p>Monica Vitti, I’m so sorry to say, sadly passed away in 2022 at the tender age of <strong>92 years old</strong>. Yes. And her Antonioni’s films mentioned earlier were filmed back in the 1960s -in black and white- when her career and looks were at their peak.</p><h4>Wan 2.1 as a Time Machine</h4><p>The photo below shows the real Monica Vitti in her 30s. The animated GIF above, at the top of this tutorial, also shows her in her 30s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*58MTSEhg7J7gNnV4WHZ2ig.jpeg" /><figcaption>The real Monica Vitti looking bored in a restaurant in the 1960s.</figcaption></figure><p>So in a way Wan 2.1 went back in time -some 60 years ago- and brought the Monica from the 1960s to our current era, 2025.</p><p>Unfortunately for our frustration, Monica Vitti’s fans, the 1960s actor is not longer with us, but thanks to <strong>Wan 2.1</strong> text to video model we can settle with a representation of what the AI model it thinks she looks like conditioned on a custom dataset and a text prompt: That is, it synthetically generated a short video clip conditioned by typing <em>Monica Vitti in an elegant event holding a drink , skyscrapers in the background in a night scene</em>, and a captioned image datase…</p><h4>- Wait… Wan… What?</h4><h4>- Never heard of it? It’s like OpenAI’s Sora but better (and free)</h4><p>Wan 2.1 is the new kid on the block in the Generative AI field. The coolest kid, actually. It’s a state-of-the-art <a href="https://github.com/Wan-Video/Wan2.1">text-to-video model and image-to-video model</a> realsed by Qwen (from Alibaba Group) last February 25th, 2025. It’s very powerful and, indeed, considered superior by many to OpenAI’s Sora (as of March 2025). And it’s also very accessible: it’s has a very permissible open source licence (Apache 2.0) which -translated- means is FREE TO USE!</p><h4>- Lets stick it to the man!</h4><p>Exactly ! That’s what I’m trying to say: Lets avoid closed source models like OpenAI’s Sora, yeah? <a href="https://www.thirdway.org/report/open-source-ai-is-a-national-security-imperative">Open source is better.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*Lewq6YFSDzyrRHQJ.png" /><figcaption>OpenAI’s Sam Altman (aka The Man)</figcaption></figure><h4>-Alright then!</h4><h3>This guide</h3><p>In this guide you will learn how to fine-tune Wan 2.1 (the grown up, 14 billion parameters version as opposed to the 1.3 billion one) and, once you get the checkpoints (fine-tuned versions of the model), you will be able to generate short videos using ComfyUI based on your own custom dataset of someone (with their permission obviously), objects or places. If you don’t have a custom dataset prepared, don’t worry, I have you covered (see below).</p><p>So by the end of this tutorial you should know:</p><ul><li>How to deploy a GPU instance on the cloud using RunPod.</li><li>How to set up a Wan 2.1 trainer (Diffusion-pipe)</li><li>How to run a training session using Wan 2.1 text-to-video 14b</li><li>How to generate samples using ComfyUI</li><li>How to evaluate your LoRA checkpoints</li><li>How to export generated samples to your computer from the cloud.</li><li>How to set up monitoring of your training run with Tensorboard.</li><li>How to stop and terminate a GPU instance on RunPod</li></ul><p>The fine-tune method you are going to use is called LoRA, which is very convenient (in terms of time and cost), as opposed to doing a full fine-tune (which requires a lot of compute resources, time and money).</p><p>The guide also covers inferencing in ComfyUI. By that I mean you will be able to generate images of your fine-tuned LoRA model using the most popular tool for that matter: ComfyUI by <a href="https://github.com/comfyanonymous">comfyanonymous</a>.</p><h4>Remember: You can also use this guide as a reference for fine-tuning on objects or places or styles or camera movements or….</h4><p>It really depends on your image dataset and how you caption your images. Just use your own custom dataset. I’ll explain what I mean by a dataset below.</p><p>If you want to use it for training camera movements or motion in general, you could also feed the trainer with a video dataset. Or a combination of video and images dataset.</p><h3>Prerequisites</h3><ul><li><strong>Basic Knowledge of Command Line</strong>: Familiarity with commands like cd, wget, and git.</li><li><strong>Basic Knowledge of VS Code</strong>: Or familiarity with similar IDEs.</li><li><strong>RunPod Account</strong>: Create an account on <a href="https://runpod.io/">RunPod</a> to access GPU instances.</li><li><strong>Some pocket change in your RunPod account (perhaps 5 dollars?). </strong>I mentioned earlier that we can use Wan for free, which is true, but you need a computer with GPU to run it! If you have a GPU machine sitting at home, then you could be training and generating videos “for free” (well, you still have to pay for the energy bill, right?).</li><li><strong>Some 3–6 hours of spare time</strong> for setting up, training and inferencing.</li><li>A <strong>dataset</strong> with images and captions.</li></ul><h3>Estimated cost of setting up, training and generating a bunch of videos by following this tutorial? 2 to 3 dollars</h3><p>Let say you set up, train and generate like 20-50 videos using the provided dataset (see below) in two resolutions (720p and/or 480p).</p><p>That should take anywhere between 3 to 6 hours on a A40 at a rate of $0.44/hr (based on RunPod’s costs as of March 2025). That should sum up between 2 and 3 dollars in total.</p><p>Is really not that expensive compared to generating videos on a text-to-video platform where you pay a monthly subscription with limitations of all sorts -or even worst- with heavy censoring, right?</p><h3>Step 1: Set Up a RunPod Instance</h3><h4>1.1 Launch a GPU Instance</h4><ol><li>Go to <a href="https://runpod.io/">RunPod</a>. (https://runpod.io/)</li><li>Create a new instance with the following specifications:</li></ol><ul><li><strong>Compute Power</strong>: Choose a GPU instance with 48GB or higher (e.g., A40 GPU).</li><li><strong>Storage</strong>: 150GB or more.</li></ul><p>Lets have a detailed look of how to achieve the above.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0V9wzbpXOtNks-2FwN1_UQ.png" /><figcaption>Go to Pods (left panel) then Click on “Deploy a Pod”</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F9InS22yOSAfgZcys-4nwQ.png" /><figcaption>Select a GPU device (in this example is an A40)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-kbHI0-XidChkiK7SMjJ-w.png" /><figcaption>Click on “Change Template”</figcaption></figure><h4>1.2 Use a RunPod Template</h4><p>Select the following Docker template (Dec 2025 UPDATE: Docker image might contain an outdated ComfyUIwhich includes all necessary tools (like ComfyUI, the trainer and other dependencies) pre-installed. This template will simplify setup.</p><pre>madiator2011/better-comfyui:full</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZcLHNupX12Yfa6hvv8pZLQ.png" /><figcaption>In the search field write the name of the template I gave you above and it should appear at the bottom (See where it says Community)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KPd1R8gm63RnZ3_WmiLaHg.png" /><figcaption>Once you have selected the template, click eon Edit Template</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*esOBLTbrum7TxTmljYscXg.png" /><figcaption>increase the disk size (here we are making it 150GB in size) and click on “Set Overrides”</figcaption></figure><h4>1.3 Launch pod</h4><p>Once you have selected the correct RunPod template and modified its configuration (say you made it 150GB in size) click on the “On Demand” option. Then, at the botton, click on the “Deploy-On Demand” button.</p><p>Be patient because it could take several minutes to download all files required and setup the trainer.</p><blockquote>PROTIP: It’s ok to stop and terminate a pod that is taking too long to download files, and then choose another, faster pod.</blockquote><p>You will see a bunch of log of files being downloaded. You just have to wait till the Connect button gets activated. Once it is activated, you know everything was downloaded and installed with success.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*hdwCq1Bl5j-a48p0MB1Wow.png" /><figcaption>Wait till the Connect button gets activated.</figcaption></figure><h3>Step 2: Download and Organise Required Models</h3><h4>2.1 Open VS Code</h4><p>One of the nicest things about the custom template is that once is fully installed you can open VS Code (a powerful code editor) directly on your web browser.</p><p>In order to launch it click on the Connect button. A window will pop up with two options: “Launch ComfyUI on port 3000” or “Launch VS Code on port 7777”. You will only launch VS Code by clicking on <strong>HTTP Service -&gt; 7777</strong> once it says “Ready” in green letters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eq6-6BhZNn84kfkM3vTzag.png" /><figcaption>Launching VS Code (click HTTP Service 7777)</figcaption></figure><p>With VS Code you can easily navigate the folders and files from the explorer on the located on the left panel. You can even open terminals from within as it is already connected to the GPU server via ssh (secure shell connection). If you don’t know what am I on about, don’t worry: Use VS Code as some sort of file explorer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tgJJBOjerWim6U8EW1-y8w.png" /><figcaption>VS Code opened from the RunPod console. Note that there’s still no ComfyUI folder on the left panel.</figcaption></figure><h4>2.2 Wait till ComfyUI is fully installed</h4><p>After launching VS Code you’ll notice there’s no ComfyUI folder in the left panel. You’ll have to wait till a folder named exactly “ComfyUI” shows up there. It can take up to 10 minutes. So in the meantime…</p><h4>2.3 Get the models ComfyUI and Wan 2.1 need</h4><p>Open a terminal from VS Code (click on the toggle panel little button on the top right menu) to download the three required models for both ComfyUI (a text encoder, a VAE and a Wan 2.1 for generating videos) and the whole Wan 2.1 model (which is going to be used by the diffusion-pipe trainer).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/692/1*hfZKAPmLp55xHT8P5_ASSQ.png" /><figcaption>Click on the little icon on the top right corner that when you hover over it says “Toggle Panel” (this is VS Code)</figcaption></figure><p>A terminal will open that will look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-l1MMYOe_bopZFhbmw-phg.png" /><figcaption>Notice there is a + sign on the top right of the image. Pressing it will launch another terminal.</figcaption></figure><blockquote><em>PROTIP: To add another terminal, just click on the + button.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/157/1*ApAYrsuguUO2gI-ebF4Jow.png" /><figcaption>You can also navigate different terminals by clicking on them.</figcaption></figure><p>Copy the following commands and paste them in the opened terminal (in Linux use CTRL+SHIFT+V to paste in terminal). To execute them press Enter in your keyboard.</p><p><strong>The first time you paste them in the VS code terminal a small window will show up asking you whether to allow or deny the operation. Click Allow.</strong></p><pre><br># ComfyUI models: text encoder, VAE and Wan2.1<br>cd /workspace/<br>wget https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors<br>wget https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/vae/wan_2.1_vae.safetensors<br>wget https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/diffusion_models/wan2.1_t2v_14B_bf16.safetensors<br>echo &quot;DONE&quot;</pre><p>In another terminal, start downloading the Wan 2.1 14B text-to-video model from the WanAI HuggingFace official repository. We need to download it for our training purposes.</p><pre># Diffusion-pipe mode: Wan2.1 T2V 14B<br>cd /workspace/<br>pip install huggingface_hub[cli]<br>huggingface-cli download Wan-AI/Wan2.1-T2V-14B --local-dir wan2.1<br>echo &quot;DONE&quot;</pre><p>Since it’s a very big model, it would take some time to download. Be patient.</p><blockquote>PROTIP: If you see the last download progress bar entirely full but with no apparent shell prompt, press enter and it might show up as it is hidden behind the bars.</blockquote><h4><strong>2.4 Once ComfyUI is ready</strong></h4><p>Once ComfyUI has finished setting up you’ll see its folder in VS Code’s left panel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U2DB5ND9oD0FxF35HEgrwQ.png" /></figure><p>Great! It’s time to move the downloaded <strong>text encoder, VAE and Wan 2.1 models </strong>ComfyUI needs<strong> </strong>to their respective folders:</p><pre>cd /workspace/<br># Create folders (props to redditor Wrektched for noticing they weren&#39;t being created) <br>mkdir -p ComfyUI/models/text_encoders/<br>mkdir -p ComfyUI/models/diffusion_models/<br>mkdir -p ComfyUI/models/vae/<br># Move downloaded folders to their respective ones in ComfyUI<br>mv umt5_xxl_fp8_e4m3fn_scaled.safetensors /workspace/ComfyUI/models/text_encoders/<br>mv wan_2.1_vae.safetensors /workspace/ComfyUI/models/vae/<br>mv wan2.1_t2v_14B_bf16.safetensors /workspace/ComfyUI/models/diffusion_models/</pre><p>If some of these commands fails it probably because you didn’t wait for ComfyUI to be fully installed OR one of the models hasn’t been fully downloaded yet. So wait till the ComfyUI folder appears on the left panel and/or wait till the models are fully downloaded.</p><h3>Step 3: Clone and Set Up the Diffusion-Pipe Repository (our Wan 2.1 trainer)</h3><p>Diffusion-pipe is an open source, popular trainer used to fine-tune models such as Wan 2.1. The Wan 2.1 trainer module is rather new so feel free to contribute to it on its <a href="https://github.com/tdrussell/diffusion-pipe/">official repository</a>.</p><p>In another terminal in VS Code, clone the Wan 2.1 trainer repository and install its dependencies:</p><pre># Install the trainer<br>cd /workspace/<br>git clone --recurse-submodules https://github.com/learn2train/diffusion-pipe<br>cd /workspace/diffusion-pipe/<br>pip install -r requirements.txt<br><br># Install flash attention separately<br>pip install flash-attn</pre><p>Remember you started downloading the Wan model in <strong>step 2.3</strong> ? Well, if it’s already downloaded then move it to its final destination folder. You have to wait for the model to finish downloading.</p><pre>cd /workspace/<br>mv wan2.1/ /workspace/diffusion-pipe/models</pre><h3>Step 4: Prepare Your Dataset</h3><p>Create directories for your dataset and upload your data:</p><pre>cd /workspace/diffusion-pipe/<br>mkdir -p data/input data/output</pre><p><strong>Upload your images and corresponding text files</strong> to /workspace/diffusion-pipe/data/input.</p><p>You can can simply use the VS Code file explorer and right-click on the input folder, click Upload, and then select your images and texts for upload.</p><h4>Haven’t prepared your dataset yet?</h4><p>A dataset is said to be prepared when every image has a caption describing it. It may or may not include other configuration settings read by the training application.</p><p>The image dataset I use in this tutorial consist of about 100 images with 100 text files.</p><p>This is an example of how images and caption files are formatted in our dataset:</p><ul><li>image1.jpg</li><li>image1.txt &lt;= Same filename as the image file</li></ul><p>The text file image1.txt contains the caption describing image-name1.jpg, say, for example: a photo of a Monica Vitti wearing a floral crown and holding a bouquet of flowers.</p><p>Note that the trainer can take the usual image files such as <strong>jpg, jpeg, png, webp, etc</strong>. It is supposed to take video as well (but I haven’t tested it yet).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nxrRi9nL1JtNLmwiNs1VAg.png" /><figcaption>A preview of our own Monica Vitti’s curated dataset (captioned using OpenFlamingo 4B)</figcaption></figure><h4>Ah! You can’t even bother to prepare a dataset!?</h4><p>Well, you can also use our prepared Monica Vitti’s dataset by downloading from our Google Drive and into the GPU instance.</p><p>Copy the following commands and, in another terminal, paste them to get our training dataset extracted in the target folder.</p><pre># Upgrade system<br>apt update<br># Install unzip<br>apt install -y unzip<br># Install gdowm client (Google Drive client)<br>pip install --upgrade gdown<br># Download dataset from Google Drive<br>gdown 1odCUGHaS4rE6khWnBP6M0QUFZpjfevVI -O dataset.zip<br># Unzip dataset in target directory<br>unzip dataset.zip -d /workspace/diffusion-pipe/data/input/</pre><h4><strong>Google Drive</strong></h4><p>The <em>pip install — upgrade gdown</em> command will install the Google drive command-line utility that would let you download files from public Google Drives.</p><p>Once all the images and corresponding text files are extracted in the diffusion-pipe data/input folder, you can actually see the images and their respective captions (text files).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*55XCQmFDubUNAzmZzJekDg.png" /><figcaption>Once dataset is downloaded and extracted in the target folder, you should see both images and captions</figcaption></figure><h3>Step 5: Configure Training Parameters</h3><p>Modify the training configuration file (wan_video.toml)to suit your needs:</p><pre># This command will modify the wan_video.toml config file.<br># It will modify the number of epochs to run from 1000 to 100<br>sed -i &#39;s|^epochs = 1000|epochs = 20|&#39; /workspace/diffusion-pipe/examples/wan_video.toml</pre><p>In the example above, I edited the number of epochs to be run in total using the `sed` command.</p><p>Of course, you don’t have to use the `sed` command, you can also edit the <strong>wan_video.toml </strong>manually by opening directly with VS Code. It is located in the <strong>diffusion-pipe/examples</strong> folder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mt4ZnaIPeHJAy8Ah0xhzuw.png" /><figcaption>You can also directly edit the wan_video.toml file located at the examples folder.</figcaption></figure><h4>Can’t wait hours to see some results?</h4><p>Then I would highly recommend to edit the wan video configuration file to tell the trainer how often it will save epochs.</p><p>For example, saving a LoRA every 2 epochs:</p><pre># diffusion-pipe/examples/wan_video.toml<br><br>save_every_n_epochs = 2</pre><p>If you want to resume training in a later date or time, make sure to save checkpoints (see below for more information on resuming training). Example:</p><pre># diffusion-pipe/examples/wan_video.toml<br><br># Can checkpoint the training state every n number of epochs or minutes. Set only one of these. You can resume from checkpoints using the --resume_from_checkpoint flag.<br>#checkpoint_every_n_epochs = 1<br>checkpoint_every_n_minutes = 30</pre><p>WARNING: Training takes time. It’s like making a stew.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4Ckw5kUEL696SuZ4.jpg" /><figcaption>If you’re good at preparing a great stew, you’ll probably be a very successful trainer. Maybe.</figcaption></figure><p>If you really can’t wait an hour or so to see results, then you’ll probably see your LoRA models performing poorly. That is, you’ll get bad results from any of your text prompts, no matter how much you change your ComfyUI node settings.</p><p>Training it’s a bit like preparing a stew. It wouldn’t probably taste good if you take it out of the oven just after 20 minutes. It would probably taste much better if you’ve had waited hours. But, also, if you leave the stew in the oven for too many hours, you run the risk of burning everything. Same with training an AI model. Wait too long, and your LoRA would start overfitting (i.e. outputting bad results or incapable of generalising). Wait too little and your LoRA won’t have learned anything from your image dataset.</p><h4>Evaluate your LoRAs (saved epochs) by sampling videos</h4><p>I’d strongly recommend to wait more than an hour to see some decent results. Try to generate samples at every saved epoch, though, to see how the training is progressing by checking the samples. If you keep sampling videos using with the same prompt and seed (set the KSampler’s control after generate to “fixed”) and generation settings you should see whether your LoRAs are learning your subject or not.</p><p>Keep evaluating your video samples until you are satisfied.</p><blockquote><em>PROTIP: Remember that you can sample videos while running a training session since we are using a very powerful GPU device (the A40 48GB)</em></blockquote><h3>Step 6: Run the Training</h3><h4>6.1 Run Tmux</h4><p>It’s important to set up a tmux (or Screen) session to run the training process just in case your internet connection fails (you really DO NOT want your training to be lost after running some hours!)</p><pre>tmux new -s training<br></pre><p>If you find yourself having to resume the tmux session run this command:</p><pre>tmux attach-session -t training</pre><h4>6.2 Run the trainer</h4><p>Once you are in tmux mode you can now safely start your training run.</p><pre><br>cd /workspace/diffusion-pipe/<br>NCCL_P2P_DISABLE=&quot;1&quot; NCCL_IB_DISABLE=&quot;1&quot; deepspeed --num_gpus=1 train.py --deepspeed --config examples/wan_video.toml</pre><p>This command starts the training with the specified configurations.</p><p>At the start of training, you’ll see some initial output with some warnings (ignore them), the caching of your dataset, and finally, the step counting.</p><p>So sit back and relax. Training takes its time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oDyE4g0hrMFCEOYza8f96w.gif" /><figcaption>Monica Vitti is already in chapter 7 of her book and your training session hasn’t even reached epoch 2.</figcaption></figure><h4>6.3 Move generated LoRA checkpoints to ComfyUI’s LoRAs folder</h4><p>Depending on how you configured the trainer, it could take few minutes or hours for a checkpoint to be saved in the diffusion-pipe output folder.</p><p>When an epoch is saved you will see a new folder with the name “epoch-{NUM-EPOCH}” format. In the image you can see an <strong>epoch10</strong> folder was created inside the <strong>output/20250228_04-50–50</strong> folders. Inside this folder there are two important files:</p><pre># Contents of diffusion-pipe/data/output/20250228_04-50-50/epoch10<br>adapter_config.json<br>adapter_model.safetensors</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0jS73Tz3hX3MuLmWEEvpPA.png" /><figcaption>The two important files are inside the epoch10 folder.</figcaption></figure><p>Using VS Code, copy these two files and paste them inside the <strong>ComfyUI/models/loras</strong> folder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/446/1*cKcFpUND1cDb70rFCR5MRA.png" /><figcaption>This is where the adapter_config.json and adapter_model.safetensors go.</figcaption></figure><p>Once you have those two important files (<strong>adapter_config.json and adapter_model.safetensors</strong>) inside the <strong>ComfyUI/models/loras</strong><em> </em>folder<em>, </em>for practical purposes, rename the <strong>adapter_model.safetensors</strong> to <strong>epoch{NUM-EPOCHS}.safetensors. </strong>Example, if your epoch is number 10, then rename adapter_model.safetensors to epoch10.safetensors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/1*B7BGOwIXLbfFXKyWux3z-A.png" /><figcaption>In this example, we renamed it to epoch10.safetensors</figcaption></figure><h3>Step 8: Launch ComfyUI</h3><h4>8.1 Start the Comfy server</h4><p>In another terminal in VS Code, navigate to the ComfyUI directory and start the server:</p><pre>cd /workspace/ComfyUI/<br>python main.py</pre><h4>8.2 Launch ComfyUI</h4><p>After some time a you will see a <strong>http://localhost:8188</strong> or <a href="http://localhost:8188/"><strong>http://127.0.0.1:8188</strong></a><strong> </strong>url. Click it to open ComfyUI in the web browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kfYWi89ZF9PXSIIm2-tJ_Q.png" /><figcaption>Click the URL with CTRL+click to launch ComfyUI in the web brower.</figcaption></figure><h4>8.3 Update ComfyUI</h4><p>On the ComfyUI click on the Manager button:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/382/0*AKTct015t22QdUJU.png" /><figcaption>Click on the Manager button</figcaption></figure><p>Once the <strong>Comfy Manager Menu</strong> window opens up click on the <strong>Update ComfyUI </strong>button to update it to its latest version. Wait for some minutes till you get the message it’s updated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iM61nB7NOaK21bxo.jpg" /><figcaption>ComfyUI Manager Menu</figcaption></figure><p>You’ll see a message once it has finished with updating. Proceed to click on the red <strong>Restart</strong> button.</p><h3>Step 9: Generate Videos</h3><h4>9.1 Restart ComfyUI to load epochs</h4><p>Once you have an epoch saved in ComfyUI’s models/loras folder, restart the ComfyUI by going to the terminal that is running the current ComfyUI server and pressing CTRL+C to force it to shutdown.</p><p>Once is closed, run the ComfyUI server again.</p><pre>cd /workspace/ComfyUI/<br>python main.py</pre><p>And wait till you see the <a href="http://localhost:8188/">http://localhost:8188</a> or <a href="http://127.0.0.1:8188/">http://127.0.0.1:8188</a> URL in the console. Once the URL is there click on it to open a new ComfyUI in a new web browser tab.</p><h4>9.2 Import the Workflow</h4><p>Go to the following URL:</p><p><a href="https://raw.githubusercontent.com/learn2train/ComfyUI-Batch-Video-Generator-Wan-2.1/refs/heads/main/workflows/mvitti.json">https://raw.githubusercontent.com/learn2train/ComfyUI-Batch-Video-Generator-Wan-2.1/refs/heads/main/workflows/mvitti.json</a></p><p>You should see something like this in your web browser:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uZqMpDXAHN1qsUQu3Jfbeg.png" /><figcaption>This is the workflow to be pasted in ComfyUI’s canvas.</figcaption></figure><p>What you seeing is the ComfyUI template of the Monica Vitti workflow. You are going to use this workflow as a base. You can edit it later.</p><p><strong>Copy the contents</strong> of the entire page with CTRL+A (in Windows/Linux) or Edit &gt; Select All in your web browser.</p><p>Go back to ComfyUI.</p><p><strong>Paste the workflow</strong> directly anywhere on a clear space in the ComfyUI canvas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W5CQayAQtWTkFN4z_LvPgQ.png" /><figcaption>Just pate the workflow anywhere in an open space in the ComfyUI canvas.</figcaption></figure><p>If after pasting it you have an alert popping up related to EmptyHunyuanLatentVideo, just <strong>ignore it </strong>by closing it. It should be gone once ComfyUI is updated and restarted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XgaptAJ7zEWBY0lWXy6CVg.png" /><figcaption>No worries. Close the warning window.</figcaption></figure><p>Now you’ll see our workflow template ready to generate videos of Monica Vitti (or your subject if you are using your own dataset) from a epoch10.safetensors LoRA model -as you can see in the <strong>LoraLoaderModelOnly </strong>node. Change it to the actual epoch (or epochs) you saved in the ComfyUI loras folder by clicking on <strong>lora_name</strong> in the <strong>LoraLoaderModelOnly node </strong>and selecting the epoch of your choice from the pull down menu. For example, if you saved your LoRA checkpoint as epoch10.safetensors, then select epoch10.safetensors:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/379/1*45kzxND8wKmjxJRIFhttRg.png" /><figcaption>Lora Model selector node</figcaption></figure><h4>9.3 Write Your Prompt</h4><p>On the CLIP Text Encode (Positive Prompt) enter your prompt. For example:</p><p><strong>Prompt:</strong> “Monica Vitti on a vintage car looking glamorous.”</p><p>Leave the Negative prompt as it is. You can check on Google Translate what all those Chinese characters mean.</p><p><strong>Spoiler alert</strong>: It’s an incantation to summon demons from hell!!!</p><p>Ok, no, not really. It’s just basically a bunch of words and phrases used to avoid getting “ugly” videos. You can delete them if you want to see if it makes a difference during your testings.</p><h4><strong>9.4 Generate videos!</strong></h4><p>You’ve arrived at the most exciting stage! Now you are ready to generate. Simply click <strong>Run or Queue button </strong>in the ComfyUI canvas to generate a video based on your prompt and node settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/379/1*eJ-DSjDgFN_nTyn83JBdBA.png" /><figcaption>ComfyUI’s Queue button. The number 1 is number of videos to generate.</figcaption></figure><p>Note that the first time you generate a video, the VAE, the text encoder and the Wan 2.1 models used would be loaded into the memory. Further video generations will be faster (as they won’t load all these models again).</p><p>For example, if you rented the A40 and trained on our Monica Vitti dataset it should take about 15–30 minutes to generate the first video with the default ComfyUI workflow settings. But remember, next generation should be much faster.</p><h4>Reduce image resolution or KSampler steps for quicker generations</h4><p>If you want to speed up the time it takes to generate a video, reduce the image resolution (e.g. from 720x1280 to 480x832), or decrease the number of KSampler steps (e.g. from 30 to 20).</p><p><strong>WARNING</strong>: Reducing the resolution or number of steps will most likely have a negative effect on the quality of the generated video.</p><h4>Experiment with the workflow configurations</h4><p>Feel free to edit the settings of the <strong>KSampler</strong> node, the <strong>SaveAnimatedWEBP</strong> node and the <strong>LatentImage</strong> node. These settings are pretty much the same of those of Stable Diffusion models like the number of steps, seed, CFG, image resolution, etc. Just be mindful that changes in this node would impact the quality and speed of video generation.</p><h3>Step 10: Export Your LoRAs and Samples</h3><p>Once you are satisfied with your results, <strong>DO NOT FORGET </strong>to export your LoRAs (and configuration file) into your local computer:</p><pre>cd /workspace/ComfyUI/models/loras/<br>runpodctl send adapter_config.json<br>runpodctl send epoch10.safetensors<br>runpodctl send epoch20.safetensors</pre><p><strong>runpodctl</strong></p><p>In the above example we are downloading to our local computer two checkpoints (epoch 10 and epoch 20) and a configuration file using RunPod’s own <strong>runpodctl</strong> command-line client. You could also use <strong>runpodctl</strong> transfer to yourself your generated videos by sending the entire ComfyUI output folder (e.g. runpodctl send output/).</p><p>The <strong>runpodctl send</strong> command line will generate a phrase which you need to paste in a terminal in your local computer/laptop. This should start the connection and transfer of the data from the GPU server into your computer/laptop.</p><p>Of course, you could also send the LoRA models (aka checkpoints, LoRAs), samples, config files, etc. to yourself via the VS Code by right clicking on the folder or file and then clicking Download.</p><h3>OPTIONAL: Resuming training</h3><p>The main reason why we save checkpoints is to resume training in the future.</p><p>To resume from a checkpoint run the trainer with the <em>— resume_from_checkpoint</em> flag and the name of checkpoint folder (i.e. the datetime folder inside the ‘/workspace/diffusion-pipe/data/output’ directory). Example:</p><pre># Resuming a training session from a checkpoint<br>cd /workspace/diffusion-pipe/<br>NCCL_P2P_DISABLE=&quot;1&quot; NCCL_IB_DISABLE=&quot;1&quot; deepspeed --num_gpus=1 train.py \<br>--deepspeed \<br>--config examples/wan_video.toml \<br>--resume_from_checkpoint 20250406_10-15-29<br></pre><h4>What is this checkpoint folder anyway?</h4><p>The datetime folder contains a Deepspeed checkpoint.</p><p>From the diffusion-pipe trainer’s <a href="https://github.com/tdrussell/diffusion-pipe">documentation</a>:</p><blockquote>Deepspeed checkpoints are in directories named like global_step1234. These checkpoints contain all training state, including weights, optimizer, and dataloader state, but can’t be used directly for inference. The saved model directory will have the safetensors weights, PEFT adapter config JSON, as well as the diffusion-pipe config file for easier tracking of training run settings.</blockquote><h3>OPTIONAL: Monitoring your training run</h3><p>We can use Tensorboard to monitor the training in real time.</p><p>To do so execute the following commands in a new terminal:</p><pre>cd /workspace/diffusion-pipe/data/<br>tensorboard --logdir output --host 0.0.0.0</pre><p>This will invoke tensorboard to read the events file inside the output folder.</p><p>Tensorboard will be ready when you see the <a href="http://0.0.0.0:6006/">http://0.0.0.0:6006/</a> url in the terminal. Click on the URL and you should see the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/1*q3UD5wwpT9J7pov2NFKKdg.png" /></figure><p>Click the Open button and Tensorboard will launch in a new tab.</p><p>The available metrics are <strong>train/epoch_loss</strong> and <strong>train/loss</strong>.</p><p>They are fun to watch but they aren’t very useful for short training sessions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bpJt8560rI164wLG2mYBTw.png" /><figcaption>Tensorboard dashboard</figcaption></figure><h3>P.S. Terminate your RunPod instance</h3><p>Once you have finished your training and/or generated enough videos to your liking, stop the instance and destroy it. Fail to do so will ran out your credit balance. Remember, you are renting a GPU computer and they charge you by the hour (in fact, by the minute).</p><p>In case you didn’t get the memo. Remember this:</p><blockquote>Make sure to STOP and then TERMINATE the instance.</blockquote><p>And once terminated be aware of this:</p><blockquote>Everything you did on the GPU will be lost forever.</blockquote><p>So you better export all your outputs before terminating it.</p><p>Anyway, lets see how to stop and terminate a GPU Instance on RunPod.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Psi9yqoH22lexVEzIl4PxA.png" /><figcaption>Go to your Pod and click on the stop button.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xHZ9ykZ0yVVOWdJXp4-ZGA.png" /><figcaption>Confirm you want to stop the Pod.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Psi9yqoH22lexVEzIl4PxA.png" /><figcaption>You thought it was enough to stop the pod? Well, think again. You still need to terminate it! So click on the Rubish Bin to Terminate instance.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HpfpAw2HsJHoPvZf9XOYbg.png" /><figcaption>Confirm you want to terminate the Pod so you won’t incur in further charges.</figcaption></figure><h3>Conclusion</h3><p>You’ve successfully fine-tuned a text-to-video model using Wan 2.1 and generated your own videos using ComfyUI. This process allows you to adapt the model to specific datasets, enabling creative and personalised video generation. For further improvements, consider adjusting hyper-parameters or experimenting with different datasets.</p><p>Anyway, happy video generation! I hope your AI videos end up in a modern art gallery in the near future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZO0HfZ6fMVIZMwTMjxBqKw.gif" /><figcaption>Monica Vitti is not impressed by your AI art video installation.</figcaption></figure><h4>And that’s it!</h4><p>If you have a comment or question, spotted an error, disagree with something, have a better idea, please share it with the rest of the class below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6f0b334ab79" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Transcribing Your Podcast for Free (up to 45,000 minutes!): A Step-by-Step Tutorial]]></title>
            <link>https://learn2train.medium.com/step-by-step-tutorial-transcribing-your-podcast-for-free-up-to-45-000-minutes-7f6c68300833?source=rss-6d74d24f9c45------2</link>
            <guid isPermaLink="false">https://medium.com/p/7f6c68300833</guid>
            <category><![CDATA[api]]></category>
            <category><![CDATA[deepgram]]></category>
            <category><![CDATA[transcription]]></category>
            <category><![CDATA[podcast]]></category>
            <category><![CDATA[speech-recognition]]></category>
            <dc:creator><![CDATA[learn2train.org]]></dc:creator>
            <pubDate>Sun, 18 Feb 2024 13:08:43 GMT</pubDate>
            <atom:updated>2024-02-22T14:25:30.753Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Two podcasters" src="https://cdn-images-1.medium.com/max/1024/1*IN-_Uibl1R2LHu_ydRyoQA.jpeg" /><figcaption>Two podcasters — Made with Stable Cascade</figcaption></figure><h3>Enhance Your Podcast with Deepgram</h3><p>Welcome to our step-by-step tutorial on transcribing your podcast for free using the <a href="https://github.com/roperi/deepgram-transcription-processor">Deepgram transcription processor</a> a Python script designed to simplify the transcription process. Transcribing your podcast episodes can greatly enhance accessibility, improve SEO, and increase engagement with your content.</p><p>In this tutorial, we’ll guide you through the process of setting up your Deepgram API key, installing the necessary dependencies, transcribing your podcast episodes, and reviewing the transcription results. Whether you’re a seasoned podcaster looking to improve your workflow or a newcomer eager to make your content more accessible, this tutorial has got you covered. Let’s get started!</p><h4>Requirements</h4><ul><li>Python 3.x</li><li>Deepgram account (it’s free, no credit card required, plus they give you $200 in credit and up to 45,000 minutes of transcription)</li></ul><h3>1. Setup Deepgram API Key</h3><p>Before using the command-line tool you need to obtain an API key from <a href="http://deepgram.com/">Deepgram’s</a> transcription service. New accounts receive $200 in credit (up to 45,000 minutes) for free, with no credit card required. Follow these steps to set up your API key:</p><ul><li>Sign Up/Login to Deepgram: If you haven’t already, sign up for a Deepgram account or log in to your existing account <a href="https://www.deepgram.com/">here</a>.</li><li>Obtain API Key: Once logged in, navigate to your account settings or API dashboard to obtain your API key.</li><li>Create .env File: Create a file named .env in the root directory of the Deepgram Transcription project.</li><li>Set API Key in .env: Open the .env file and add the following line, replacing YOUR_API_KEY with your actual API key:</li></ul><pre># .env<br><br>DG_API_KEY=&#39;YOUR-API-KEY&#39;</pre><h3>2. Clone the Repository</h3><p>First, clone the Deepgram Transcription repository to your local machine. Open a terminal or command prompt and run the following command:</p><pre>git clone https://github.com/roperi/deepgram-transcription-processor.git</pre><p>3. Install Dependencies</p><p>Navigate to the cloned repository directory and install the required dependencies using pip:</p><pre>cd deepgram-transcription-processor/<br>pip install -r requirements.txt</pre><h3>4. Prepare Your Podcast Audio</h3><p>Ensure that your podcast episodes are available as audio files in a supported format such as WAV or MP3. If your episodes are hosted online, make sure you have the direct URLs to the audio files.</p><p>I use <a href="http://podcastindex.org/">podcastindex.org</a> to find audio URLs for the podcasts I’m interested in transcribing.</p><h3>5. Transcribe Your Podcast Episodes</h3><p>Use thetranscribe.py script to transcribe your podcast episodes. Here are some usage examples:</p><ul><li>Transcribe a local audio file:</li></ul><pre>python transcribe.py --name &quot;Podcast Episode About Cats&quot; --input /path/to/your/podcast_episode.wav</pre><ul><li>Transcribe audio from a URL:</li></ul><pre>python transcribe.py --name &quot;Podcast Episode About Dogs&quot; --input https://example.com/your/podcast_episode.mp3</pre><h3>6. Review Transcription Results</h3><p>After running the transcription script, you’ll find the transcription results saved in JSON and TXT formats in the output directory. You can review the transcribed text and extract summary, paragraphs, and topics from the results.</p><pre># Output:<br><br>output/Podcast Episode About Dogs__transcription.json<br>output/Podcast Episode About Dogs__paragraphs.txt<br>output/Podcast Episode About Dogs__summary.txt<br>output/Podcast Episode About Dogs__topics.txt</pre><h3>7. Further Processing (Optional)</h3><p>Depending on your needs, you can further process the transcription results. For example, you can convert the JSON transcription to a more readable format, such as plain text or HTML, or integrate the transcription into your podcast website or platform.</p><h3>9. Share Your Transcriptions</h3><p>Once you’re satisfied with the transcriptions, you can share them with your audience! Consider adding transcriptions to your podcast website or including them in episode descriptions to improve accessibility and reach a wider audience.</p><h4>And that’s it!</h4><p>If you have a comment or question, spotted an error, disagree with something, have a better idea, please share it with the rest of the class below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f6c68300833" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Preparing and creating a dataset for training Stable Diffusion]]></title>
            <link>https://learn2train.medium.com/preparing-and-creating-a-dataset-for-training-stable-diffusion-98bdf4854dde?source=rss-6d74d24f9c45------2</link>
            <guid isPermaLink="false">https://medium.com/p/98bdf4854dde</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[fine-tuning]]></category>
            <category><![CDATA[training]]></category>
            <category><![CDATA[dataset]]></category>
            <dc:creator><![CDATA[learn2train.org]]></dc:creator>
            <pubDate>Sat, 17 Feb 2024 13:02:37 GMT</pubDate>
            <atom:updated>2024-02-17T13:02:37.611Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/362/1*F1j7C17TYKHGUxrJ9ljUeQ.jpeg" /><figcaption><em>“An angel with white wings and a white dress standing in a field of white flowers with a cloudy sky in the background” — Image generated with our </em><a href="https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64"><em>tutorial’s</em></a><em> fine-tuned model. Caption was auto-generated with OpenFlamingo 3B.</em></figcaption></figure><h3>Unlocking the Power of Stable Diffusion: A Comprehensive Guide to Dataset Preparation</h3><p>In this segment of the <a href="https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64">Training Stable Diffusion In A Low-Cost Cloud GPU: A Step-by-Step Guide for Non-Technical Folks</a> series, we’ll explore the critical role of captioning in image selection and dataset preparation for fine-tuning the Stable Diffusion base model. As we venture into leveraging machine learning’s creative capacities, we recognize the pivotal importance of data curation in achieving exceptional outcomes.</p><p><strong>Understanding the Influence of Captioning: Enhancing Model Performance</strong></p><p>In this article, we’ll delve into how meticulous data selection and preparation, particularly in captioning, significantly impact the model’s performance and generalization capabilities. By carefully curating captions that accurately represent the target domain and ensuring their alignment with the selected images, we lay the groundwork for a robust and effective fine-tuning process.</p><p><strong>Fostering Diversity for Enhanced Generalization</strong></p><p>It’s essential to grasp that a well-prepared dataset, enriched with diverse and informative captions, not only enhances the model’s learning potential but also fosters generalization. This diversity empowers the model to generate compelling and coherent visual content across a wide range of applications, ensuring its versatility and adaptability in real-world scenarios.</p><p><strong>Setting up a Cloud-Based Environment for Upload and Data Storage</strong></p><p>Once you have curated and preprocessed the dataset for fine-tuning the Stable Diffusion model, the next step is to create a suitable cloud-based environment to facilitate the efficient upload and storage of the prepared data.</p><p>Setting up the cloud environment typically involves selecting a cloud service provider (e.g., AWS, Google Cloud, Microsoft Azure) that offers the necessary infrastructure and storage capabilities for handling your dataset size. This environment ensures that your data is readily accessible for the fine-tuning process and provides the computational resources required to train the model effectively.</p><p>Note that in our <a href="https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64">step-by-step</a> guide we decided to go for a low-cost GPU cloud solution instead of relying on the big three mentioned above.</p><h3><strong>Selecting and Preparing Images</strong></h3><p><strong>EveryDream2</strong></p><p>As per our <a href="https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64">step-by-step guide</a> we are using EveryDream2 to train our custom Stable Diffusion model. This trainer excels in fine-tuning models for different scales. Prioritizing versatility with a focus on image and caption pairs, it diverges from Dreambooth by recommending ground truth data, eliminating the need for regularization images. This makes EveryDream 2 a flexible and effective choice for seamless Stable Diffusion training.</p><p><strong>Number of images</strong></p><p>You should probably start with less than 100 images, until you get a feel for training. EveryDream2 is capable of handling hundreds of thousands of images.</p><p><strong>Image size and quality</strong></p><p>EveryDream2 supports `.jpg`, `.jpeg`, `.png`, `.bmp`, `.webp`, and `.jfif` image formats.</p><p>The author of EveryDream2 recommends images to have <strong>at least</strong> 1 megapixel (ex 1024x1024, 1100x900, 1300x800, etc). That being said, technology continues to advance rapidly. EveryDream2 has no problem handling 4K images, so it’s up to you to pick the appropriate trade-off with disk and network costs.</p><p>Bare minimum recommended size is 512x512. Scaling images up is not a great idea though, though it may be tolerable as a very small percentage of your data set. If you only have 512x512 images, don’t try to train at 768.</p><p>Use high quality, in-focus, low-noise, images, capturing the concept(s) under training with high fidelity wherever possible.</p><p><strong>Cropping and Aspect Ratios</strong></p><p>You can crop your images in an image editor <strong>if it highlights the concept under training</strong>, e.g. to get good close ups of things like faces, or to split images up that contain multiple characters.</p><p><strong>You do not need to crop to square.</strong></p><p>Aspect ratios between 4:1 and 1:4 are supported; the trainer will handle bucketing and resizing your images as needed.</p><p>It is OK to use a full shot of two characters in one image and also a cropped version of each character separately, but make sure every image is captioned appropriately for what is actually present in each image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*O9GAaTOY_LNNDuP4g3nYdw.png" /><figcaption>Three ways of cropping an image</figcaption></figure><p>The scene above offers various cropping possibilities to highlight different aspects. You could opt for the orange box to focus solely on the man in the checker jacket, the blue box to encompass both men, or encompass the entire scene.</p><p>Each cropping choice corresponds to a distinct caption:</p><ul><li>For instance, with the orange crop, the caption might read: “A man in a checker jacket, hands in pockets, stands on a sidewalk with a pond in the background, wearing sunglasses and looking sideways.”</li><li>In contrast, the blue crop could yield a caption like: “Two men walk together on a sidewalk, with a pond and trees visible in the background.”</li><li>Finally, selecting the purple crop could result in a caption such as: “Two men stroll in Washington, D.C., facing each other, with the Washington Monument and Lincoln Memorial reflection pool in the background, reminiscent of a film still.”</li></ul><p>However, it’s imperative to ensure that even after cropping and captioning, the remaining image retains sufficient resolution. If the original image lacks resolution, smaller crops may prove inadequate and should be avoided.</p><h4><strong>Caption Design</strong></h4><p>Captioning is pivotal in text-to-image model training, profoundly influencing the quality and coherence of generated images. Findings indicate that the presence or absence of captions greatly impacts output quality. While uncaptioned images may seem less impactful initially, they hold potential in larger-scale training. The type of captions, whether keyword-based or descriptive, dictates consistency and flexibility. Precise captions outperform inaccurate ones, though sometimes sacrificing length and detail. Strategic activation phrase placement enhances consistency but may limit flexibility. These insights highlight the critical role of captions in achieving optimal text-to-image model performance.</p><p><strong>Caption</strong></p><p>A caption consists of a main prompt, followed by one or more comma-separated tags.</p><p>For most use cases, use a sane English sentence to describe the image. Try to put your character or main object name close to the start.</p><p><strong>If you are training on images of humans, there is little benefit in using “unique” names most of the time</strong>. Don’t worry so much about using a “rare” toking, or making up gibberish words. Just try generating a few images using your concept names, and make sure there are no serious conflicts.</p><p><strong>Use normal names for people and characters, such as “mickey mouse” or “john david washington” instead of making up weird names like “cldstrf” or “jhndvdwshtn”. There’s no need for this and it just makes inference less natural and shifts a burden on the user to remember magic names.</strong></p><p>Those training anime models can use booru tags as well using other utilities to generate the captions.</p><p><strong>Styles</strong></p><p>For style, consider adding a suffix on the caption that describes the style. Examples would be “by rembrandt” or “in the style of minecraft” at the end of the caption. This will help the model learn recall style at inference time so you can style other subjects you did not train with the style. You may also consider “drawing of” or “painting of” at the start of the caption when appropriate.</p><p>Consider also including a style tag as above if you are training anything besides photos. For instance, if you are training a few characters from a video game you can consider “mickey mouse holding a buster sword, screenshot from dragon ball for ps4” if you wish to capture the “style” of the game render along with the characters.</p><p><strong>Context</strong></p><p>Include the surroundings and context in your captions. Ex. “mickey mouse standing on a dirt path in rio city slums district” Again, this will allow you to recall the “dirt path in rio city slums district” style at inference time, and will even pick up on pieces of that like “rio city” (if enough samples are present with similar words) as a style or scenery you can apply later. This can extract additional value from your training besides just the character.</p><p>Also consider some basic mention of pose. ex. “mickey mouse sitting on a blue wooden bench in front of a concrete wall” or “piglet holding his fist in front of his face with an angry look on his face, looking at the camera.” Captions can capture value not only for the character’s look, but also for the pose, the background scene, and the camera angle. You can be creative here, there is a lot of potential!</p><h3><strong>Constructing a dataset</strong></h3><p><strong>What is a dataset?</strong></p><p>A dataset consists of image files coupled to captions and other configuration.</p><p>You are welcome to use any folder structure that makes sense for your project, but you should know that there are configuration tricks that rely on data being partitioned into folders and subfolders.</p><p><strong>Assigning captions</strong></p><p>by Filename</p><p>The simplest and least powerful way to caption images is by filename. The name of the file, without extension, and excluding any characters after an _ (underscore).</p><pre>a photo of paddington bear, sitting on a green armchair_1.jpg<br>a photo of paddington bear, laying down_1.jpg<br>a photo of paddington bear, laying down_2.jpg</pre><p>by Caption file</p><p>If you are running in a Windows environment, you may not be able to fit your whole caption in the file name. Instead you can create a text file with the same name as your image file, but with a `.txt` or `.caption` extension, and the content of the text file will be used as the caption, <strong>ignoring the name of the file</strong>.</p><p>by Caption yaml</p><p>You can capture a more complex caption structure by using a `.yaml` sidecar instead of a text file. Specifically you can assign weights to tags for shuffling.</p><p>The format for `.yaml` captions:</p><pre>main_prompt: a photo of ted bennet<br>tags:<br>- &quot;sitting on a green armchair&quot; # The tag can be a simple string<br>- tag: &quot;wearing a tuxedo&quot; # or it can be a tag string</pre><p>Assigning captions to entire folders</p><p>In some cases it may make sense to add the same tag to all images in a folder. You can place any configuration that should apply to all images in a local folder (ignoring anything in any subfolders) by adding a file called `local.yaml` to the folder. In this file you can, for example, add:</p><pre>tags:<br>- tag: &quot;in the style of xyz&quot;</pre><p>And this tag will be appended to any tags specified at the image level.</p><p>If you want this tag, or any other configuration, to to apply to images in subfolders as well you can create a file called `global.yaml` and it will apply to all images in the local folder <em>and</em> to any images in any subfolders, recursively.</p><p><strong>Other image configuration</strong></p><p>In addition to captions, you can also specify the frequency with which each image should show up in training (`multiply`), or the frequency in which the trainer should be given a flipped version of the image (`flip_p`), or the frequency in which the caption should be dropped completely focusing the training on the image alone, ignoring the caption (`cond_dropout`).</p><p>For simple cases you can create a file called `multiply.txt`, `flip_p.txt`, and/or `cond_dropout.txt`, containing the single numeric value for that configuration parameter that should be applied to all images in the local folder.</p><p>Alternatively you can add these properties to any of the supported `.yaml` configuration files, image-level, `local.yaml`, and/or `global.yaml`</p><pre>main_prompt: a photo of paddington bear<br>tags:<br>- sitting on a green armchair<br>multiply: 2<br>flip_p: 0.5<br>cond_droput: 0.05</pre><h3><strong>Empowering AI: The Key Role of Data Preparation</strong></h3><p>In summary, the meticulous selection, preparation, and construction of a dataset are fundamental to successful machine learning endeavors. By curating relevant images and incorporating diverse captions, we empower our models to learn effectively and generate compelling visual content across various applications. As we advance in the field of machine learning, the importance of data preparation remains paramount for unlocking the full potential of AI-driven solutions.</p><h4>And that’s it!</h4><p>If you have a comment or question, spotted an error, disagree with something, have a better idea, please share it with the rest of the class below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98bdf4854dde" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fine-Tuning Stable Diffusion With A Curated Dataset (Step-by-Step Guide)]]></title>
            <link>https://learn2train.medium.com/a-step-by-step-interactive-guide-on-fine-tuning-stable-diffusion-with-a-curated-dataset-c327135e9158?source=rss-6d74d24f9c45------2</link>
            <guid isPermaLink="false">https://medium.com/p/c327135e9158</guid>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[photography]]></category>
            <category><![CDATA[hugging-face-tutorials]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[dataset]]></category>
            <dc:creator><![CDATA[learn2train.org]]></dc:creator>
            <pubDate>Sat, 16 Dec 2023 21:26:40 GMT</pubDate>
            <atom:updated>2024-02-19T17:08:58.192Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/578/0*S612RYIT-ckJcrXq" /><figcaption>Image generated with a fine-tuned Stable Diffusion base model</figcaption></figure><h4>Introduction</h4><p>This article is based on the Jupyter notebook included in the <a href="https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64">Training Stable Diffusion with a low-cost Cloud GPU</a> guide.</p><p>You can read this tutorial and copy+paste the code into a new Jupyter Notebook. Or you can read the tutorial while having the original Jupyter notebook open in another browser tab.</p><p>The notebook can be found <a href="https://github.com/learn2train/l2t-sd/blob/main/notebooks/Train_photographer.ipynb">here</a>.</p><p>If you are only interested in the dataset you can get it with <a href="https://pypi.org/project/gdown/"><strong>gdown</strong></a>:</p><pre>pip install --upgrade gdown<br>gdown 1Ifk07HeqxHfCCOCvb5oDF-cdxfkfsuq- -O dataset.zip</pre><p>If you have your own dataset and want to use it instead make sure to read <a href="https://learn2train.medium.com/preparing-and-creating-a-dataset-for-training-stable-diffusion-98bdf4854dde">Preparing and Creating the Dataset</a> article first.</p><h4>About this guide</h4><p>In this guide, we’ll cover the following topics in an interactive way:</p><ol><li><strong>Fine-tuning a Stable Diffusion base model with a custom dataset</strong>.</li><li><strong>Upload the training dataset to the cloud GPU instance</strong>.</li><li><strong>Start the training job</strong></li><li><strong>Monitor your sample generations in Weights &amp; Biases (W&amp;B)</strong>. W&amp;B is a free tool used to visualise machine learning experiments. No installation is required as it will be run from a standalone webpage.</li><li><strong>Training is done</strong></li><li><strong>Upload the fine-tuned models to Hugging Face (optional)</strong> so you can re-use them later. Hugging Face is an open-source community for AI experts and enthusiasts. It’s free to use!</li><li><strong>Evaluate the fine-tuned checkpoints</strong> to asses its performance.</li><li><strong>Terminate the GPU instance</strong>. Avoid incurring charges by destroying the GPU instance.</li></ol><h3>Requirements</h3><p>This interactive tutorial assumes you have:</p><ul><li>Installed EveryDream2 trainer from a Docker container. If you haven’t already, please review <a href="https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64">this guide</a>.</li><li>A GPU with 24GB of VRAM. If you don’t posses such compute power you could always rent a low-cost Cloud GPU instance for few USD cents per hour (see <a href="https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64">this guide</a>)</li><li>A basic understanding of how Jupyter Notebooks work (check this <a href="https://jupyter.org/try-jupyter/notebooks/?path=notebooks/Intro.ipynb">cool demo</a> if you’re unfamiliar)</li><li>A reliable internet connection.</li><li>An updated browser such as Chrome, Safari, Firefox, etc.</li><li>Time to train (it will take about 20 minutes to train the dataset)</li></ul><h3>1. Fine-tuning a Stable Diffusion base model with custom data</h3><h4>Fine-tuning using a photographer’s image dataset</h4><p>In this tutorial we will train a Stable Diffusion 1.5 base model in the style of a photographer that it doesn’t knows very well. By feeding the model with a photographer’s image dataset, it should be able to generate better pictures in the style of the photographer than what the base model currently does.</p><p>Important: This guide is for educational purposes only. Train with your own artwork once you understand the effects of training a curated dataset.</p><h4>Bella Kotak</h4><p>In this tutorial, we will fine-tune a Stable Diffusion text-to-image model using Bella Kotak recent artwork. She’s is an award-winning UK-based photographer with a strong, distinctive style.</p><p>Check <a href="https://www.instagram.com/bellakotak">her Instagram account</a>…and be amazed!</p><h4>Before fine-tuning the base model</h4><p>Hope you checked her portfolio because you need to know how much the base model needs to learn in order to be able to generate decent-looking synthetic images in her artistic style.</p><p>Since the base model wasn’t trained with enough pictures of her artwork, it fails to portray her unique artistic vision. So if we prompt the base Stable Diffusion 1.5 model with <strong>“a black and white photo of a woman wearing a floral crown and holding a bouquet of flowers in the style of Bella Kotak”</strong>, the base model will struggle to generate a picture that represents her style, or even follow the prompt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/0*ddv31DHWhDmxGnpO" /><figcaption>Prompt: “a black and white photo of a woman wearing a floral crown and holding a bouquet of flowers in the style of Bella Kotak” (base Stable Diffusion model)</figcaption></figure><h4>After fine-tuning the base model</h4><p>Thankfully by fine-tuning the base Stable Diffusion model using captioned images, the ability of the base model to generate better-looking pictures based on her style is greatly improved. And even the prompt is better followed.</p><p>Image below was generated on a fine-tuned Stable Diffusion 1.5 model. It has the same prompt, seed, resolution, and CFG values as the image above!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/0*Xoz-e-hRuRtrZOSg" /><figcaption>Prompt: “a black and white photo of a woman wearing a floral crown and holding a bouquet of flowers in the style of Bella Kotak” (fine-tuned Stable Diffusion model)</figcaption></figure><p>As you can see, it’s not perfect -for one thing it’s not exactly black and white- but yet the differences between the non-fine tuned model and the fine-tuned one are rather noticeable. That’s the power of training a Stable Diffusion base model with a custom dataset.</p><h3>2. Upload the training dataset</h3><h4>Download and extract the dataset</h4><p>We are going to download an already prepared training dataset into our GPU instance.</p><p>A dataset is said to be prepared when every image has a caption describing it. It may or may not include other configuration settings read by the training application.</p><p>Our image dataset contains ~200 images, ~200 text files, and 1 tag configuration file (global.yaml) that adds a suffix tag to each text file (in this case appends the phrase in the style of Bella Kotak to each caption description for each image).</p><p>This is an example of how images and caption files are formatted in our dataset:</p><ul><li>image-name_001.jpg</li><li>image-name_001.txt &lt;= Same filename as the jpg file</li></ul><p>The text file image-name_001.txt contains the caption describing image-name_001.jpg, say, for example: a photo of a woman wearing a floral crown and holding a bouquet of flowers in the style of Bella Kotak.</p><p>Open a new Jupyter Notebook and name it. Add a new cell and copy and paste the following code into it, then run it. It will download a public ZIP file from Google Drive, extract it and store it in the <strong>input</strong> subfolder.</p><pre>import os<br>import zipfile<br><br># Install gdown (to be able to download files from Google Drive)<br>!pip install gdown<br><br># Download dataset<br>os.makedirs(&#39;input&#39;, exist_ok=True)<br>path_to_dataset = &quot;input/dataset.zip&quot;<br><br>if not os.path.exists(path_to_dataset):<br>    !gdown 1Ifk07HeqxHfCCOCvb5oDF-cdxfkfsuq- -O input/dataset.zip<br>else:<br>    print(f&quot;Already downloaded `{path_to_dataset}`&quot;)<br><br># Unzip dataset into &#39;input&#39; folder<br>with zipfile.ZipFile(path_to_dataset, &#39;r&#39;) as zip_ref:<br>    zip_ref.extractall(&#39;input/dataset&#39;)<br><br># Remove zip file<br>os.remove(path_to_dataset)<br><br># List input directory<br>%ls input/<br><br>print(&#39;Done&#39;)</pre><h3>3. Start the training job</h3><p>Once our training images and their captions are inside the <strong>input folder</strong> we are ready to train the model.</p><h4>Training configuration</h4><p>We will override the following default training settings:</p><ul><li><strong>project name</strong>: “sd1_kotak” &lt;= Name of the project. It is convenient to name it in a way that identifies it from other training sessions.</li><li><strong>data_root</strong>: “input” &lt;= Folder location of the training images</li><li><strong>max epochs</strong>: 60 &lt;= An epoch refers to the one entire passing of training images through the trainer. We are doing 60 entire passes.</li><li><strong>batch size</strong>: 6 &lt;= Determines the amount of images that are going to be trained every epoch</li><li><strong>sample steps</strong>: 80 &lt;= Determines how frequently samples are generated. In this case we will generate them every 80 steps.</li><li><strong>save every n epochs</strong>: 20 &lt;= Checkpoints will be saved every 20 epochs (since we are doing 60 epochs, we will end with 3 checkpoints)</li><li><strong>save ckpt dir</strong>: “ouput” &lt;= Folder location of the saved checkpoints</li><li><strong>zero_frequency_noise_ratio</strong>: 0.04 &lt;= This will make dark scenes look more realistic</li><li><strong>optimizer_config</strong>: optimizer-photo.json &lt;= We add an optimiser config file to get better results</li><li><strong>cond_dropout</strong>: 0.0 &lt;= This will prevent the trainer learning images without captions</li></ul><p>The are more configurations not show here. For a detailed explanation of each check <a href="https://github.com/victorchall/EveryDream2trainer/blob/main/doc/TRAINING.md">EveryDream 2 documentation</a>.</p><h4>Download the optimizer configuration file</h4><p>Copy and paste the following code into a new cell and run it. It should get the optimiser configuration settings to improve our training.</p><pre>!wget https://raw.githubusercontent.com/learn2train/l2t-sd/main/notebooks/optimizer-photo.json</pre><h4>Setup Weights &amp; Biases for monitoring</h4><p>Create a new cell to paste your API token in it. With W&amp;B you can track your training progress. If you don’t have an account, you can create one for free at <a href="https://wandb.ai/site">https://wandb.ai/site</a>. This is a free online logging utility.</p><p>You can get your API key from your <a href="https://wandb.ai/settings">User Settings</a>.</p><pre>wandb_token = &quot;PUT-YOUR-W&amp;B-API-KEY-HERE&quot;</pre><p>Make sure to run the cell above with your W&amp;B API key. It should look something like:</p><blockquote>wandb_token = “28d37291d39f337237291d39f391d39f3”</blockquote><h4>Running a training session</h4><p>To start training copy the code below and paste it in a new cell. As soon as you run it will start printing its log. Keep scrolling down to monitor the current status of the training session.</p><pre># Get the wandb token<br>wandb_settings = &quot;&quot;<br>if wandb_token:<br>  !rm /root/.netrc<br>  !wandb login $wandb_token<br>  wandb_settings = &quot;--wandb&quot;<br><br># Start the training <br>%run train.py --resume_ckpt &quot;learn2train/stable-diffusion-v1-5&quot; \<br>$wandb_settings \<br>--project_name &quot;sd1_kotak&quot; \<br>--data_root &quot;input&quot; \<br>--max_epochs 60 \<br>--sample_steps 80 \<br>--batch_size 6 \<br>--save_every_n_epochs 20 \<br>--zero_frequency_noise_ratio 0.04 \<br>--cond_dropout 0.0 \<br>--optimizer_config optimizer-photo.json \<br>--save_ckpt_dir &quot;output&quot;</pre><p><strong>IMPORTANT: If you see messages with a red backround, IGNORE THEM as they are only warning messages</strong></p><p>The training time takes about 20 minutes on a RTX 3090 with 24GB of VRAM.</p><p>While you wait for the Training completed message, watch the samples being generated in Weight &amp; Biases dashboard.</p><h3>4. Watch your samples in Weights &amp; Biases (W&amp;B) while the trainer is running</h3><h4>W&amp;B dashboard</h4><p>Go to the <a href="https://wandb.ai/home">W&amp;B dashboard</a> in another browser tab. You will see your training session in your projects.</p><p>Click on your training session to check your samples. They should give you an idea how good/bad your model learning progress is going. You should stop the training once you are satisfied with the results you are seeing.</p><p>Samples come in three. That is because each sample generated uses different CFG values (1, 4 and 7).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*utP0YN9F9gecNSaY5X3AnA.png" /><figcaption>W&amp;B Dashboard — A training run page</figcaption></figure><h3>5. Training is over</h3><p>Once the training is done you should see the following messages:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/0*JiD9XMeqs5y11pab" /><figcaption>EveryDream2 output</figcaption></figure><p>That was it! The base model has been updated and you are now left with some checkpoints (safetensors files).</p><p>You could now upload your checkpoints into your Hugging Face repository for further re-use.</p><p>If you decide to save the checkpoints to your computer - I do NOT recommend it as it could take a lot of time- go to the file explorer, double click the <strong>output </strong>folder, right-click a checkpoint and click <strong>Download.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cqeNeGrMK8C5JxFQ1nuj9Q.png" /></figure><h3>6. Upload your checkpoints to Hugging Face (optional but highly recommended)</h3><p>If you aren’t saving your checkpoints to your computer, you could save them to your Hugging Face repository instead. That way you can easily re-use or share them.</p><h4>Get a Hugging Face token</h4><p>If you haven’t got one yet, have a look at <a href="https://learn2train.medium.com/a-step-by-step-guide-to-host-stable-diffusion-checkpoints-on-hugging-face-for-free-2098d0c18a01">How to Host Stable Diffusion Checkpoints on Hugging Face for Free</a> to get your own token.</p><h4>Log-in into your account</h4><p>Copy and paste the code below in a new cell , and run it. It will prompt your <strong>Hugging Face write token. </strong>Put it to log into your account to be able to upload data into your repo.</p><pre># Log in to Hugging Face<br><br>from huggingface_hub import notebook_login, hf_hub_download<br>import os<br>notebook_login()</pre><h4>Upload checkpoints to your model repository</h4><p>Make sure you are <strong>logged in</strong> to Hugging Face by running the above login cell first.</p><p>The cell below will upload one or more checkpoints to your personal Hugging Face repository. You should already be authorized to Huggingface by token if you used the download/token cells above.</p><p>When you run the cell below, a box will show up and you need to <strong>CLICK</strong> to select which .safetensors files are marked for upload. This allows you to select which ones to upload. If you don&#39;t click of the ckpts, nothing will happen.</p><p>You will also be required to fill-in your username and your repository name:</p><ul><li>Hugging Face username: <strong>your username</strong> (look for it in your <a href="https://huggingface.co/settings/account">Hugging Face account page</a>).</li><li>Hugging Face repository name: <strong>your repository name</strong></li></ul><p><strong>WARNING</strong></p><p><strong>If your Hugging Face account is brand new upload only 3 checkpoint files</strong>. For safety reasons, Hugging Face limits the amount of files a new user can make. If you try to upload more than 3 checkpoint files you’ll probably get a warning tell you to wait 24 hours to keep uploading.</p><pre># Run this cell after reading the instructions of the cell above <br><br>import glob<br>import os<br>from huggingface_hub import HfApi<br>from ipywidgets import *<br>all_ckpts = [f for f in glob.glob(&quot;output/*.safetensors&quot;)]<br>  <br>ckpt_picker = SelectMultiple(options=all_ckpts, layout=Layout(width=&quot;600px&quot;)) <br>hfuser = Text(placeholder=&#39;Hugging Face username&#39;)<br>hfrepo = Text(placeholder=&#39;Hugging Face repository name&#39;)<br>api = HfApi()<br>upload_btn = Button(description=&#39;Upload&#39;)<br>out = Output()<br>def upload_ckpts(_):<br>    repo_id=f&quot;{hfuser.value or hfuser.placeholder}/{hfrepo.value or hfrepo.placeholder}&quot;<br>    with out:<br>        if ckpt_picker is None or len(ckpt_picker.value) &lt; 1:<br>            print(&quot;Nothing selected for upload, make sure to click one of the ckpt files in the list, or, you have no ckpt files in the current directory.&quot;)<br>        for ckpt in ckpt_picker.value:<br>            print(f&quot;Uploading to HF: huggingface.co/{repo_id}/{ckpt}&quot;)<br>            response = api.upload_file(<br>                path_or_fileobj=ckpt,<br>                path_in_repo=ckpt,<br>                repo_id=repo_id,<br>                repo_type=None,<br>                create_pr=1,<br>            )<br>            display(response)<br>        print(&quot;DONE&quot;)<br>upload_btn.on_click(upload_ckpts)<br>box = VBox([ckpt_picker, HBox([hfuser, hfrepo]), upload_btn, out])<br>display(box)</pre><h4>Save the uploads to your model repository</h4><p>To actually save the uploaded checkpoints into your repository, go back to your Hugging Face model repository and click the <strong>Community</strong> tab. You’ll see a list of all the uploaded checkpoints. Go one by one and click <strong>Merge</strong> to save them to your model repository:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kio6bn2iYiYuOmi1" /><figcaption>Merging checkpoints (saving them to Hugging Face)</figcaption></figure><h3>7. Evaluate the fine-tuned model</h3><h4>Test inference on your checkpoints</h4><p>To recap: Training is over and you are left with model checkpoints. These checkpoints are updated fine-tuned models saved at different times during the training session.</p><p>The main idea here is to evaluate each of your checkpoints to find the ones that generate the output you like the most.</p><p>Running the bit of code in a new cell will display a mini text-to-image generator like this one:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YHH2Eg9gJGOVHO4hlXQKlg.png" /></figure><p>You can select any saved checkpoint -or all of them- and set inference parameters such as prompt, steps, CFG, resolution and seed.</p><p>Have fun!</p><pre>from ipywidgets import *<br>from IPython.display import display, clear_output<br>import os<br>import gc<br>import random<br>import torch<br>import inspect<br><br>from torch import autocast<br>from diffusers import StableDiffusionPipeline, AutoencoderKL, UNet2DConditionModel, DDIMScheduler, DDPMScheduler, PNDMScheduler, EulerAncestralDiscreteScheduler<br>from transformers import CLIPTextModel, CLIPTokenizer<br><br>checkpoints_ts = []<br>for root, dirs, files in os.walk(&quot;.&quot;):<br>        for file in files:<br>            if os.path.basename(file) == &quot;model_index.json&quot;:<br>                ts = os.path.getmtime(os.path.join(root,file))<br>                ckpt = root<br>                checkpoints_ts.append((ts, root))<br>checkpoints = [ckpt for (_, ckpt) in sorted(checkpoints_ts, reverse=True)]<br>full_width = Layout(width=&#39;600px&#39;)<br>half_width = Layout(width=&#39;300px&#39;)<br>checkpoint = Dropdown(options=checkpoints, description=&#39;Checkpoint:&#39;, layout=full_width)<br>prompt = Textarea(value=&#39;a photo of &#39;, description=&#39;Prompt:&#39;, layout=full_width)<br>height = IntSlider(value=512, min=256, max=768, step=32, description=&#39;Height:&#39;, layout=half_width)<br>width = IntSlider(value=512, min=256, max=768, step=32, description=&#39;Width:&#39;, layout=half_width)<br>cfg = FloatSlider(value=7.0, min=0.0, max=14.0, step=0.2, description=&#39;CFG Scale:&#39;, layout=half_width)<br>steps = IntSlider(value=30, min=10, max=100, description=&#39;Steps:&#39;, layout=half_width)<br>seed = IntText(value=-1, description=&#39;Seed:&#39;, layout=half_width)<br>generate_btn = Button(description=&#39;Generate&#39;, layout=full_width)<br>out = Output()<br>def generate(_):<br>    with out:<br>        clear_output()<br>        display(f&quot;Loading model {checkpoint.value}&quot;)<br>        actual_seed = seed.value if seed.value != -1 else random.randint(0, 2**30)<br>        text_encoder = CLIPTextModel.from_pretrained(checkpoint.value, subfolder=&quot;text_encoder&quot;)<br>        vae = AutoencoderKL.from_pretrained(checkpoint.value, subfolder=&quot;vae&quot;)<br>        unet = UNet2DConditionModel.from_pretrained(checkpoint.value, subfolder=&quot;unet&quot;)<br>        tokenizer = CLIPTokenizer.from_pretrained(checkpoint.value, subfolder=&quot;tokenizer&quot;, use_fast=False)<br>        scheduler = DDIMScheduler.from_pretrained(checkpoint.value, subfolder=&quot;scheduler&quot;)<br>        text_encoder.eval()<br>        vae.eval()<br>        unet.eval()<br>        text_encoder.to(&quot;cuda&quot;)<br>        vae.to(&quot;cuda&quot;)<br>        unet.to(&quot;cuda&quot;)<br>        pipe = StableDiffusionPipeline(<br>            vae=vae,<br>            text_encoder=text_encoder,<br>            tokenizer=tokenizer,<br>            unet=unet,<br>            scheduler=scheduler,<br>            safety_checker=None, # save vram<br>            requires_safety_checker=None, # avoid nag<br>            feature_extractor=None, # must be none of no safety checker<br>        )<br>        pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)<br>        <br>        print(inspect.cleandoc(f&quot;&quot;&quot;<br>              Prompt: {prompt.value}<br>              Resolution: {width.value}x{height.value}<br>              CFG: {cfg.value}<br>              Steps: {steps.value}<br>              Seed: {actual_seed}<br>              &quot;&quot;&quot;))<br>        with autocast(&quot;cuda&quot;):<br>            image = pipe(prompt.value, <br>                generator=torch.Generator(&quot;cuda&quot;).manual_seed(actual_seed),<br>                num_inference_steps=steps.value, <br>                guidance_scale=cfg.value,<br>                width=width.value,<br>                height=height.value<br>            ).images[0]<br>        del pipe<br>        gc.collect()<br>        with torch.cuda.device(&quot;cuda&quot;):<br>            torch.cuda.empty_cache()<br>            torch.cuda.ipc_collect()<br>        display(image)<br>            <br>generate_btn.on_click(generate)<br>box = VBox(<br>    children=[<br>        checkpoint, prompt, <br>        HBox([VBox([width, height]), VBox([steps, cfg])]), <br>        seed, <br>        generate_btn, <br>        out]<br>)<br><br>display(box)</pre><h3>8. Terminate your Cloud GPU instance once you are done</h3><p>If you are using a cloud GPU instance, don’t forget to terminate/destroy your cloud GPU instance once you are done evaluating your checkpoints, otherwise you will be still charged.</p><p>Note that once you terminate your cloud GPU instance Jupyter Lab will stop working, and if you want to use it again you’ll have to start a new training session on a new GPU instance, and start all over again.</p><h4>And that’s it!</h4><p>If you have a comment or question, spotted an error, disagree with something, have a better idea, please share it with the rest of the class below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c327135e9158" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create a Free Hugging Face Repo to Host Stable Diffusion Checkpoints (Step-by-Step Guide)]]></title>
            <link>https://learn2train.medium.com/a-step-by-step-guide-to-host-stable-diffusion-checkpoints-on-hugging-face-for-free-2098d0c18a01?source=rss-6d74d24f9c45------2</link>
            <guid isPermaLink="false">https://medium.com/p/2098d0c18a01</guid>
            <category><![CDATA[free]]></category>
            <category><![CDATA[checkpoints]]></category>
            <category><![CDATA[data-storage]]></category>
            <category><![CDATA[hugging-face]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <dc:creator><![CDATA[learn2train.org]]></dc:creator>
            <pubDate>Sat, 16 Dec 2023 17:01:53 GMT</pubDate>
            <atom:updated>2024-02-10T17:09:01.005Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>What is Hugging Face?</strong></h4><p>Hugging Face is an open-source and platform provider of machine learning technologies. It is free to use and it also provides free storage up to 100GB. It’s a very popular and, in fact, there are about one million AI repositories (datasets, models, spaces) on Hugging Face at the moment.</p><h4><strong>Use Hugging Face to Host Fine-Tuned Checkpoints for Free</strong></h4><p>When you finish training Stable Diffusion model you are normally left with one or more checkpoints. Checkpoints are large files that contain the model’s updated weights. They are important because they are, in fact, your fine-tuned models at different stages of training.</p><p>Since they are large files one practical way with dealing with them is to keep them in your own Hugging Face model repository for fast data transfer in and out of the cloud platforms. For this tutorial, instead of keeping them in your computer, you will upload them to your Hugging Face model repository (aka repo).</p><p>Using Hugging Face repositories brings other advantages too besides offering free data and file storage. You can share your models, showcase your work to others, re-use, or even further re-train your checkpoints.</p><h4><strong>Sign up to open a Hugging Face account</strong></h4><ol><li>Go to <a href="https://huggingface.co/">https://huggingface.co</a> and click on the Sign Up button at the top right:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dxblSzRZb7pUil_c6v_UaQ.png" /></figure><p>2. Provide your <strong>email </strong>and desired <strong>password:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QEQJLWHzYIHeHiolaYj0zw.png" /></figure><p>3. Provide <strong>more account details:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gqlIw7PCOF9LSYmR8hz31g.png" /></figure><p>4. After confirming your account you’ll see this <strong>welcome home page:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g6R8dATjt3V1giZc4XwMaw.png" /></figure><h4><strong>Enable uploading</strong></h4><p>To upload files to Hugging Face you need to get an access token. So go to your <a href="https://huggingface.co/settings/tokens">Hugging Face’s Access Tokens page</a> and click the “New Token” button below the “User Access Tokens” box.</p><ol><li>From the homepage, click on <strong>Settings </strong>on the left panel:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-aH7ed_9D8rAo7O1qJY-2Q.png" /></figure><p>2. Then click <strong>Access token </strong>on the left panel.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6X1Vu94OxFdMIHZUlbTugA.png" /></figure><p>3. Once in the Access Tokens section click on the <strong>New token</strong> button below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PiaSV1hLkT-kIwxP4G52zQ.png" /></figure><p>4. When the “Create a new access token” window pops-up, fill the form with the following:</p><ul><li>Name: <strong>your-username</strong></li><li>Role: <strong>Write</strong></li></ul><p>And click “Generate a token” to create your access token. <strong>Save it and keep it at hand.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dCujk9qD1rkqZS2FbBCDvg.png" /></figure><h4><strong>Create repository</strong></h4><p>Hugging Face offers free storage for your files and models. For the purposes of this tutorial, you will upload your checkpoints to your own repository for quick and easy data transfer.</p><ol><li>Click your in avatar on the top left and then <strong>New Model:</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6jXxYP04uapqM6eu7vhLzA.png" /></figure><p>2. Then fill in the form fields with the following:</p><ul><li>Model name: <strong>your-model-name</strong></li><li>Licence: <strong>leave it blank (you can change it later)</strong></li><li>Private or Public: <strong>Public</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yIBM2KlxCUlWDxkDPamTRg.png" /><figcaption>Creating a new model repository</figcaption></figure><p>Make sure you write down the repo name for future use.</p><p>That’s it for Hugging Face. You are now ready to <a href="https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64">prepare your training application</a> and start uploading fine-tuned checkpoints to your repo!</p><h4>And that’s it!</h4><p>If you have a comment or question, spotted an error, disagree with something, have a better idea, please share it with the rest of the class below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2098d0c18a01" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Training Stable Diffusion In A Low-Cost Cloud GPU: A Step-by-Step Guide for Non-Technical Folks]]></title>
            <link>https://learn2train.medium.com/a-step-by-step-guide-for-non-technical-folks-on-training-stable-diffusion-with-a-low-cost-cloud-gpu-344c6b250d64?source=rss-6d74d24f9c45------2</link>
            <guid isPermaLink="false">https://medium.com/p/344c6b250d64</guid>
            <category><![CDATA[vastai]]></category>
            <category><![CDATA[cloud-gpu]]></category>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[everydream]]></category>
            <dc:creator><![CDATA[learn2train.org]]></dc:creator>
            <pubDate>Fri, 15 Dec 2023 18:44:01 GMT</pubDate>
            <atom:updated>2024-02-17T13:06:35.998Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*utP0YN9F9gecNSaY5X3AnA.png" /><figcaption>Samples being generated in the W&amp;B dashboard</figcaption></figure><h3>Introduction</h3><p>Welcome to this step-by-step guide for non-technical users, focusing on training Stable Diffusion with a low-cost Cloud GPU using <a href="https://github.com/victorchall/EveryDream2trainer">EveryDream 2</a>, a well-documented, robust Stable Diffusion trainer that lets you train multiple concepts at once, and/or small or large scale datasets containing hundreds of thousands of images.</p><h4>Less than 1 dollar</h4><p>In this interactive guide you will learn how to train a Stable Diffusion with a prepared custom dataset on a low-cost Cloud GPU for less than 1 dollar. The training time takes about 20 minutes on a RTX 3090 24GB, and should cost ~30–50 cents.</p><h4>Guide Sections</h4><p>This guide includes the following sections:</p><ol><li>Requirements</li><li>Creating a Vast.ai account</li><li>Configuring a Docker image for EveryDream 2</li><li>Configure and manage a GPU instance</li><li>Setting up Weights &amp; Biases for monitoring</li><li>Setting up Jupyter Lab to open a Jupyter notebook</li><li>Terminating the GPU instance</li></ol><h3><strong>Requirements</strong></h3><p>This guides assumes you have:</p><ul><li>Very little or no technical knowledge</li><li>A dataset of captioned images. If you haven’t got one check <a href="https://learn2train.medium.com/preparing-and-creating-a-dataset-for-training-stable-diffusion-98bdf4854dde">here</a> first.</li><li>A reliable internet connection.</li><li>An updated browser such as Chrome, Safari, Firefox, etc.</li><li>A valid bank card with money to top up some credit in your cloud GPU account.</li><li>Time to set up the training application (it should take less than 30 minutes)</li></ul><h3><strong>Create a cloud GPU account</strong></h3><h4>Introduction</h4><p>To train an AI Generative model, a powerful GPU is crucial. Instead of costly options from major providers like Amazon AWS or Google Cloud, consider budget-friendly alternatives like Vast.ai and Runpod.io.</p><p>Vast.ai offers cost-effective GPU rentals through a straightforward process — sign up, add a card, click RENT. Runpod.io also gives you access to low-cost GPU rentals, and makes it very easy for you to start, run, and manage a GPU instance.</p><p>While you are free to choose any other Cloud GPU platform of your liking, this guide uses an on-demand, verified GPU rental from Vast.ai for a seamless training session. We chose Vast.ai because it’s the cheapest of the two options, and offers ore control and a wide-range variety of GPU rental options that cover almost any compute need.</p><p>If you choose to rent other cloud GPU platform, like Runpod.io, you can still use this guide. Just make sure to setup the Docker container as seen in the instructions below, and open the ports for .</p><h4>Steps for creating a Vast.ai account</h4><ol><li>Go to <a href="https://www.vast.ai/">https://vast.ai</a> and click where it says <strong>CONSOLE</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ccHaXaHLoNVYUWXPQx19kQ.png" /><figcaption>vast.ai homepage</figcaption></figure><p>2. Click on <strong>SIGN IN </strong>and a window sign up form should pop-up. Fill it with your details and make sure you select account as “Client”. Click <strong>CREATE </strong>to create your Vast.ai account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uNZge_ocLjMnuD__NOrlUg.png" /><figcaption>Creating a new account</figcaption></figure><p>Once you are done you might have to confirm your email and log in again.</p><p>3.<strong> Add a card or add credit </strong>(Stripe, crypto.com, Coinbase).</p><h3><strong>Configure EveryDream 2 Docker image</strong></h3><h4>Introduction</h4><p>Think of a Docker image as a portable container that encapsulates all your training needs. EveryDream 2 is the wizard within which we use to fine-tune models effortlessly.</p><p><a href="https://github.com/victorchall/EveryDream2trainer">EveryDream 2</a>, the Stable Diffusion trainer we are using in this guide, excels in fine-tuning models for different scales. Prioritizing versatility with a focus on image and caption pairs, it diverges from Dreambooth by recommending ground truth data, eliminating the need for regularization images. This makes EveryDream 2 a flexible and effective choice for seamless Stable Diffusion training.</p><p>Now, let’s seamlessly configure this Docker image on the Vast.ai console, step by step.</p><h4>Configure the Docker image</h4><ol><li>Once you have your account log into it, enter the console and click <strong>EDIT IMAGE &amp; CONFIGURATION</strong> blue button on the left. It will open the following pop-up <strong>Docker Image Templates </strong>window</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3duzerCdVwtJCY5afKNF5w.png" /><figcaption>Configuring a Docker image template</figcaption></figure><p>2. Scroll down where it says <strong>Template Slot</strong> and click <strong>SELECT </strong>(do not click the SELECT blue button though). It should expand the window down showing you an empty form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FSsOV5OkxDNkD7yfMwI5IA.png" /><figcaption>Configuring a Docker image template</figcaption></figure><p>3. Fill the form with the following configurations for our Docker image:</p><blockquote><em>On the</em><strong><em> Docker Repository and Environment</em></strong>:</blockquote><blockquote><strong><em>Image Path/Tag: </em></strong><em>ghcr.io/victorchall/everydream2trainer:nightly</em></blockquote><blockquote><strong><em>Tag</em></strong><em>: nightly</em></blockquote><blockquote><em>On</em><strong><em> Launch Mode</em></strong><em>:</em></blockquote><blockquote><em>Select </em><strong><em>Run a jupyter python notebook (easiest)</em></strong></blockquote><blockquote><em>Check</em><strong><em> Use Jupyter Lab interface</em></strong></blockquote><p>Once you’re done scroll-down to the bottom of the form and click the <strong>SELECT &amp; SAVE </strong>blue button (don’t click the <strong>SELECT</strong> blue button).</p><p>4. You should be able to see the instance configuration you just set from the previous step:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SyYSFlfKGWiDHkRCvejtzA.png" /><figcaption>Our configured Docker image ready to go</figcaption></figure><p>Now you are ready to run and manage a GPU instance!</p><h3><strong>Configure and manage a GPU instance</strong></h3><ol><li>In the Instance Configuration panel <strong>allocate 100GB </strong>of space to the GPU instance:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A6-6M0N6E7rMtFYP7fYpXw.png" /><figcaption>Selecting disk size</figcaption></figure><p>2. Select a <strong>1x GPU</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P6GVOS7Sy_J7cm8zuKMt-g.png" /><figcaption>Selecting number of GPU units</figcaption></figure><p>3. Select a <strong>RTX 3090, RTX 4090, </strong>or an <strong>A6000 instance type</strong>. They’re all 24GB VRAM at least.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5c1MvYnUDzUm-pocjgDbcw.png" /><figcaption>Seleting GPU type</figcaption></figure><p>4. Select a <strong>region </strong>close to your current location (or to your budget):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4NLcUCQ5ks_YJYz1qJtIJw.png" /><figcaption>Selecting region</figcaption></figure><p>5. Click <strong>RENT </strong>on an instance of your choice. We recommend choosing an instance that is <strong>verified</strong>, with decent <strong>upload speed </strong>and <strong>download speed:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O17bR-dKGA46uajbIqC2-g.png" /><figcaption>Choosing an instance to rent</figcaption></figure><p>6. Once you click <strong>RENT </strong>it will start creating the GPU instance. Click <strong>INSTANCES </strong>on the left panel to see your instance creation status progress in real time as seen in the image below. Click the little black inverted triangle to expand the bottom panel. Wait till it says:</p><p><em>Status: success running ghcr.io/victorchall/everydream2trainer_cuda121/ssh</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c50P19ocwFHyFOyHt3daow.png" /><figcaption>Status bar</figcaption></figure><p><strong>NOTE</strong>: Depending on the internet speed instance creation time should takes from ~5 to ~10 minutes. If it probably encountered an error will trying to set up the docker image. You are better off destroying the instance and choose another one. Click on the <strong>TRASH CAN </strong>icon to destroy the instance. A confirmation window should pop up. Click <strong>AGREE </strong>to destroy it and try another one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-2L9g9UWsKNnA30FouLQcA.png" /><figcaption>Destroying an instance</figcaption></figure><h3><strong>Setup Weights &amp; Biases for monitoring</strong></h3><h4>Introduction</h4><p>Weights &amp; Biases (W&amp;B) is a vital FREE tool for monitoring sample generation in machine learning training. With intuitive experiment tracking, it enables real-time insights and collaboration by logging key information like hyperparameters and metrics. W&amp;B is essential for optimizing sampling strategies in ML experiments..</p><p>In our learning exercise, we’ll leverage Weights &amp; Biases to dynamically check image samples generated during training, providing a visual gauge of progress.</p><h4>Open a W&amp;B account (it’s free!)</h4><ol><li><strong>Sign up</strong>,<strong> </strong>confirm your email, and return to the site.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iTWr0T1IhNQ55YKFJaWZEg.png" /><figcaption>Signup to W&amp;B</figcaption></figure><p><strong>2. Get your API key</strong> from <a href="https://wandb.ai/settings">User Settings</a>. Once there scroll down to the Danger Zone section. Reveal your API Key and copy it to a safe place. Keep it at hand because you’ll use it later in the Jupyter Notebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dKzfz1SPLv8NdFp3oJdpsQ.png" /><figcaption>Getting your API Key</figcaption></figure><p>3. The W&amp;B dashboard should be empty. But once you are start training the run will show up like in the image below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dd-YLNZrOjmZGlqPiuP71g.png" /></figure><p>There’s nothing to show at the moment, but as soon as the training starts, training metrics will be shown for every run.</p><h3><strong>Setup Jupyter Lab to open Jupyter notebooks</strong></h3><h4>Introduction</h4><p>Jupyter Notebook is an interactive computing environment that allows users to create and share documents containing live code, equations, visualizations, and narrative text. On the other hand, Jupyter Lab is an integrated development environment (IDE) that enhances the Jupyter Notebook experience by providing a more comprehensive interface with features like a file explorer and a modular, flexible layout.</p><p>If you have interacted with <a href="https://colab.google/">Google Colab</a> before, then you probably already know what a Jupyter Notebook is as Google Colab runs on one! But if haven’t used a Jupyter Notebook before, I’d encourage you to check this <a href="https://jupyter.org/try-jupyter/notebooks/?path=notebooks/Intro.ipynb">cool demo</a> before continuing.</p><h4>Back to the Vast.ai console</h4><ol><li>Click on the <strong>Open </strong>button<strong> </strong>to open the Jupyter Notebook interface.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g9WYGAJi46sXoIHrlMqozQ.png" /></figure><p>2. Once in the Jupyter notebook interface, click on <strong>View &gt; Open JupyterLab </strong>to open it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_37_OijpwU95AcMs-7AWRg.png" /></figure><p>3. Click <strong>File</strong> on the top menu and then <strong>Open from URL</strong> to open the trainer’s notebook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xoigyNOPHgDPoiM6QLfy2Q.png" /><figcaption>Opening a Jupyter Notebook in Jupyter Lab</figcaption></figure><p>2. Copy the following URL and paste it in the Open URL field:</p><blockquote><em>https://raw.githubusercontent.com/learn2train/l2t-sd/main/notebooks/Train_photographer.ipynb</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8KwY-C-yT3MBrfoEawwFaw.png" /><figcaption>Opening a Jupyter Notebook in Jupyter Lab</figcaption></figure><p>4. After clicking OPEN a trainer notebook should show up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0_5P5TsWpiG3tU775jWRsg.png" /><figcaption>Our Jupyter Notebook</figcaption></figure><p><strong>Congratulations! You can now continue with this tutorial from the Jupyter Notebook itself</strong>. But, first, read next section to know how to terminate the GPU instance when you are done training to stop incurring charges.</p><h3><strong>Terminate the instance</strong></h3><h4>Don’t get over charged</h4><p>Destroying the instance once you are done with the training will stop being charged. To do this simply click on the <strong>TRASH CAN </strong>icon button and then click <strong>AGREE </strong>to terminate the instance.</p><p><strong>Warning</strong>: Clicking the <strong>STOP </strong>icon button won’t stop the incurring charges! You really need to terminate the instance to avoid further charges to your Vast.ai account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GodjTDjdTFIP3HHb144pFA.png" /><figcaption>Destroying an instance</figcaption></figure><p>Once you click on <strong>AGREE </strong>it will take just few seconds to destroy it and then you should be presented with an empty Instance page (unless you have other cloud GPU instances running in the background).</p><h4>And that’s it!</h4><p>If you have a comment or question, spotted an error, disagree with something, have a better idea, please share it with the rest of the class below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=344c6b250d64" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>