<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Eduardo Ortega on Medium]]></title>
        <description><![CDATA[Stories by Eduardo Ortega on Medium]]></description>
        <link>https://medium.com/@theagilecrafter?source=rss-57fab778d9e8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*MqE1WDDlxwSyhPP-DFv9jQ.jpeg</url>
            <title>Stories by Eduardo Ortega on Medium</title>
            <link>https://medium.com/@theagilecrafter?source=rss-57fab778d9e8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 10:00:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@theagilecrafter/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to add Integration Tests to our Spring Boot backend]]></title>
            <link>https://medium.com/edataconsulting/how-to-add-integration-tests-to-our-spring-boot-backend-8ab00313b6ad?source=rss-57fab778d9e8------2</link>
            <guid isPermaLink="false">https://medium.com/p/8ab00313b6ad</guid>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[junit]]></category>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[maven]]></category>
            <dc:creator><![CDATA[Eduardo Ortega]]></dc:creator>
            <pubDate>Thu, 24 Feb 2022 12:15:21 GMT</pubDate>
            <atom:updated>2022-02-24T12:15:21.584Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b1tnyXfxYKXVmmOkfDr5wg.jpeg" /></figure><p>Spring Data JPA allows us to write queries to the database in several ways. The simplest option is to use inferred queries, where Spring creates the query from the repository’s method name, but there are occasions where we need more complex queries and we must write them directly using JPQL or native SQL.</p><p>Since the possibility of making mistakes increases when using such queries, it is necessary to test them, which leads to the need to add integration tests to our project.</p><p>So, let’s check step by step how to do that using Maven.</p><h3>1. Separate Unit and Integration Tests</h3><p>I prefer to separate tests using JUnit 5 tags rather than using regular patterns to search for test files by name. So, I create two Java annotations to encapsulate the JUnit tags. (This is not strictly necessary, but makes renaming the tags easier if needed, and you can also add other annotations to all the tests which are grouped at once. There is an example of this in section 4.)</p><p>For our example, the annotations can be declared as in the first code block, and we use them as in the second.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0632d8800c3de9627c5d49e8d313097c/href">https://medium.com/media/0632d8800c3de9627c5d49e8d313097c/href</a></iframe><p>With this done, we can create different configurations in our IDE to run the tests separately and in this way speed up the development as we won’t need to wait for the integration tests to run every time we want to run the unit tests:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*d6C9rTAqIcOm_hLydjWifg.png" /><figcaption><strong>Image 1:</strong> Configuration to filter tests in IntelliJ IDEA.</figcaption></figure><h3>2. How to write the Integration Tests</h3><p>As an example, the integration test for our repository could be as shown below. Here I’m not using a custom query, but it serves well to check that the tests are being executed. To test a repository, we need to use <a href="https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.testing.spring-boot-applications.autoconfigured-spring-data-jpa"><strong>DataJpaTest</strong></a> annotation (included in <em>spring-boot-starter-test</em> artifact).This will disable full-auto-configuration and instead configure only those components relevant for JPA tests (e.g. Repository and Entity). This annotation also makes all our tests transactional, which means the transaction will roll back after each test is executed by default.</p><p>The <strong>Sql</strong> annotation is used to set up the database state before the tests. We can use it at class level (which executes the statements before each test) or at method level (which overrides the class level if used). The annotation allows us to execute statements directly (as shown in the example) or load them from a file.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba04557b732920fe36c7a6c2736c860d/href">https://medium.com/media/ba04557b732920fe36c7a6c2736c860d/href</a></iframe><h3>3. Configure the tests in Maven</h3><p>The phases of <a href="https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html">Maven’s Build Lifecycle</a> that are closely related to the tests are the following:</p><ul><li><strong>test</strong>: run tests using a unit testing framework. These tests should not require the code to be packaged or deployed.</li><li><strong>pre-integration-test</strong>: perform actions required before integration tests are executed. This may involve tasks such as setting up the required environment.</li><li><strong>integration-test</strong>: if necessary, process and deploy the package into an environment where integration tests can be run.</li><li><strong>post-integration-test</strong>: perform actions required after integration tests have been executed. This may include cleaning up the environment.</li><li><strong>verify</strong>: run any checks to verify the package is valid and meets quality criteria.</li></ul><p>From the previous list, only the <strong>test</strong> and <strong>verify</strong> phases can be executed from the command line. The phases related to integration tests will be triggered before the verification phase. As can be read in Maven’s <a href="https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#some-phases-are-not-usually-called-from-the-command-line">documentation</a>:</p><blockquote>The phases named with hyphenated-words (pre-*, post-*, or process-*) are not usually directly called from the command line. These phases sequence the build, producing intermediate results that are not useful outside the build. In the case of invoking integration-test, the environment may be left in a hanging state.</blockquote><p>To execute our tests, we need to add two different plugins to our <strong>pom.xml</strong> (notice in the code block below that I’m using <em>spring-boot-starter-parent</em> so I don’t need to specify plugin versions):</p><ul><li><strong>maven-surefire-plugin</strong>: this will run all the tests tagged as “UnitTest” (line 18).</li><li><strong>maven-failsafe-plugin</strong>: this will run all the tests tagged as “IntegrationTest” (line 25).</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5efad14912933f33eec5b21b142ef9a9/href">https://medium.com/media/5efad14912933f33eec5b21b142ef9a9/href</a></iframe><p>Now we need to start the spring-boot app to run the integration tests. To do so, we will add <a href="https://docs.spring.io/spring-boot/docs/current/maven-plugin/reference/htmlsingle/">Spring Boot Maven Plugin</a> to our pom.xml as shown below. Keep in mind that the number of milliseconds in the <strong>wait</strong> property may vary from one application to another. By default, the <strong>maxAttempts</strong> property is set to 60, so in this example, the global timeout for the application to start is 90 seconds (1500 * 60 = 90000 milliseconds). The <strong>jmxPort</strong> property specifies the port in which the app will be available. This is not required unless the default port is taken.</p><p>Also, notice lines 21 to 29: there are two ways to tell Spring what the active profile for running the integration tests is. This is important as we may need some properties to startup the application.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/49edb9b92bc3355b03eee4cdb4fc67e3/href">https://medium.com/media/49edb9b92bc3355b03eee4cdb4fc67e3/href</a></iframe><h3>4. Create application.properties file for integration-test profile</h3><p>Now that we have Maven configured, we just need to create a new properties file (<em>application-integration-test.properties</em>) and override the required properties to startup the application.</p><p>As we are going to override our datasource properties, we should modify our IntegrationTest annotation as below to ensure all classes annotated with it are executed with the right profile and datasource:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c90d39546de71c84f03aaf821d2f53b3/href">https://medium.com/media/c90d39546de71c84f03aaf821d2f53b3/href</a></iframe><p>The first thing to consider is whether we want to use an in-memory database or a <a href="https://www.testcontainers.org/">Testcontainer</a> using Docker. If we are using some kind of database migration tool such as Flyway, we should keep in mind that there is a drawback when using an in-memory database for our tests: the written SQL has to work in both (production and integration-test) databases. If our migrations use some database-specific syntax, it might not work with the H2 database. I will describe both cases.</p><h4>Case 1: In-memory database</h4><p>In this case, we will disable Flyway and use the H2 database, so we need to add the corresponding dependency to our pom.xml and <strong><em>set the scope to runtime</em></strong> so that Spring can load the driver properly.</p><p>Notice that in this example:</p><ul><li>Hibernate will use our entities to create the database schema.</li><li>Property <strong><em>DB_CLOSE_DELAY=-1</em></strong> is necessary to prevent the schema from being deleted when the statements finish executing.</li><li>Property <strong><em>DATABASE_TO_UPPER=false</em></strong> is necessary to prevent name errors when accessing fields.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/41df3cdbe638082e08f7cd7c54b672ad/href">https://medium.com/media/41df3cdbe638082e08f7cd7c54b672ad/href</a></iframe><h4>Case 2: Testcontainer database</h4><p>In this case, we will enable Flyway and use a Testcontainer, so we need to add the corresponding dependency to our pom.xml and <strong><em>set the scope to runtime</em></strong> so that Spring can load the driver properly.</p><p>Notice that in this example:</p><ul><li>Hibernate will validate the database schema that was created by Flyway.</li><li>This approach will take longer to run, since a docker container must be started.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/df95c14396e5fa09e48dc6e9622f1f40/href">https://medium.com/media/df95c14396e5fa09e48dc6e9622f1f40/href</a></iframe><h3>5. How to run the tests</h3><p>To run our tests, we can execute any of these commands: <strong><em>mvn verify</em></strong>, <strong><em>mvn install,</em></strong> or <strong><em>mvn deploy</em></strong>.</p><p>It is important to note that we don’t need to run these commands using the integration-test profile as we are changing the active profile using <em>spring-boot-maven-plugin</em>.</p><h3>Conclusion</h3><p>Dealing with integration tests can be tricky depending on how your project is built and configured, but it is important to add them as a good test harness helps to improve the quality of the project.</p><p>I hope this article will be able to help you as a starting point. Thanks for reading! 😄</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8ab00313b6ad" width="1" height="1" alt=""><hr><p><a href="https://medium.com/edataconsulting/how-to-add-integration-tests-to-our-spring-boot-backend-8ab00313b6ad">How to add Integration Tests to our Spring Boot backend</a> was originally published in <a href="https://medium.com/edataconsulting">edataconsulting</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[React components: add CSS with clean code]]></title>
            <link>https://medium.com/the-agile-crafter/react-components-add-css-with-clean-code-afcab7778804?source=rss-57fab778d9e8------2</link>
            <guid isPermaLink="false">https://medium.com/p/afcab7778804</guid>
            <category><![CDATA[react]]></category>
            <category><![CDATA[tips]]></category>
            <category><![CDATA[clean-code]]></category>
            <category><![CDATA[css]]></category>
            <dc:creator><![CDATA[Eduardo Ortega]]></dc:creator>
            <pubDate>Sun, 06 Dec 2020 17:51:15 GMT</pubDate>
            <atom:updated>2020-12-06T17:54:44.701Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y6C4nSvy2Woe0m7bWEn4BA.png" /></figure><p>Read this post in Spanish <a href="https://medium.com/the-agile-crafters-notebook-spanish/componentes-de-react-a%C3%B1adir-css-con-c%C3%B3digo-limpio-69ed94087c2">here</a>.</p><p>When it comes to designing applications with React, there are many ways to add style to components. The ones I knew so far were the following:</p><ol><li>Import a CSS file with the necessary classes and use the JSX property “className” to add them to the component.</li><li>Use the “style” property to add inline style.</li></ol><p>The disadvantage of the first option is that you have to create an additional file to create a component and it complicates the maintenance of the code a bit. Also, you have to manually avoid class name collision between components.</p><p>On the other hand, the main drawback of the second option is that you cannot copy CSS from the browser inspector and paste it into our code, or vice versa, because the “style” property of our component receives an object and the properties of the objects cannot have the “-” character in the name, so the properties must be rewritten. Also, the property value must be quoted to convert it to text. All of this slows down development because you have to retype what you test in the browser. For example, to add the following CSS snippet to a component:</p><pre>element.style {<br>  min-height: 626px;<br>  border: 1px solid red;<br>  background-color: white;<br>}</pre><p>We would have to create the following object in our code:</p><pre>const style = {<br>  minHeight: &#39;626px&#39;,<br>  border: &#39;1px solid red&#39;,<br>  backgroundColor: &#39;white&#39;<br>};</pre><p>In this post I share a clean way to style components using “styled-components”.</p><p>Styled-components is a library that creates React components with embedded styles using <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals">template literals</a>.</p><p>Some of the advantages of this library are the following:</p><ul><li>Creates class names automatically to avoid collisions.</li><li>Templates use CSS syntax.</li><li>Allows conditional rendering based on properties.</li><li>Adds vendor prefixes automatically.</li></ul><h3>Example of use</h3><p>To install the library, you have to execute the following command (the types can be omitted if TypeScript is not used, but in this case I use it):</p><pre>npm install styled-components @types/styled-components</pre><p>Imagine you want to create a list of elements and that the color of each element, as well as the cursor, changes when it is clicked on.</p><p>If the library is not used, this can be achieved by writing the following code, where I use a function to get the style using the component’s state (I write it directly in the index.tsx file for simplicity):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08ff2851621924c1d03185e2b5ec02d8/href">https://medium.com/media/08ff2851621924c1d03185e2b5ec02d8/href</a></iframe><p>And the result, after clicking on the “Three” element, would be the following image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*wZyTSImQpP38by1flfxhEA.png" /></figure><p>If we used the library, you could replace the style function with a component that can interpolate a CSS fragment depending on the value of the “selected” property. The resulting code is as follows:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e51d2f6fa48ed0aa43a342e91574415d/href">https://medium.com/media/e51d2f6fa48ed0aa43a342e91574415d/href</a></iframe><p>And the result, after clicking on the “Three” element, would be the following image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*op2oc1J9zC_1Rk0AHOJz8w.png" /></figure><p>The extension “<strong>vscode-styled-components</strong>” for Visual Studio Code is very useful to work with this library, because it adds syntax highlighting for templates and allows CSS autocompletion.</p><h3>References:</h3><ul><li><a href="https://styled-components.com/docs/basics">https://styled-components.com/docs/basics</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=afcab7778804" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-agile-crafter/react-components-add-css-with-clean-code-afcab7778804">React components: add CSS with clean code</a> was originally published in <a href="https://medium.com/the-agile-crafter">The Agile Crafter’s Notebook (english)</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker Image Optimization: from 1.16GB to 22.4MB]]></title>
            <link>https://medium.com/the-agile-crafter/docker-image-optimization-from-1-16gb-to-22-4mb-53fdb4c53311?source=rss-57fab778d9e8------2</link>
            <guid isPermaLink="false">https://medium.com/p/53fdb4c53311</guid>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[dockerfiles]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[tips]]></category>
            <dc:creator><![CDATA[Eduardo Ortega]]></dc:creator>
            <pubDate>Sun, 01 Nov 2020 13:44:52 GMT</pubDate>
            <atom:updated>2020-11-01T13:44:52.281Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*cwFmgUG9LI9s97lX.png" /><figcaption>Image from <a href="http://www.docker.com/">www.docker.com</a></figcaption></figure><p><strong>Docker </strong>is a platform for software developers and sysadmins to build, run, and share applications with containers. A <strong>container </strong>is a process that runs in an isolated environment, on its own filesystem; that filesystem is built using a <strong>docker image</strong>. Images include everything you need to run an application (compiled code, dependencies, libraries, etc.). Images are defined using a file called <strong>Dockerfile</strong>.</p><p>The terms <strong><em>dockerization </em></strong>or <strong><em>containerization </em></strong>are often used to define the process of creating a Docker container.</p><p>Containers are popular because they are:</p><ul><li><strong>Flexible</strong>: Even the most complex applications can be containerized.</li><li><strong>Lightweight</strong>: Containers share the host kernel, making them much more efficient than virtual machines.</li><li><strong>Portable</strong>: Can be compiled locally and run anywhere.</li><li><strong>Loosely coupled</strong>: Containers are encapsulated, allowing one to be replaced or upgraded without interrupting the others.</li><li><strong>Secure</strong>: Containers apply aggressive restrictions and isolations to processes without requiring any configuration on the part of the user.</li></ul><p>In this post I’m going to focus on optimizing Docker images to be lightweight.</p><p>Let’s start with an example in which we build a React application and we want to dockerize it. After running the <em>npx </em>command and creating the Dockerfile, we have a file structure like the one in <strong><em>Picture 1</em></strong>.</p><pre>npx create-react-app app --template typescript</pre><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*CwWLe_mX6YXnVlW5.png" /><figcaption>Picture 1: file structure.</figcaption></figure><p>If we build a basic Dockerfile, like the following, we end up with an image that weighs 1.16 GB:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5c99924cb1824733739949692dfbcea5/href">https://medium.com/media/5c99924cb1824733739949692dfbcea5/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*M2WiAKbrcoIwDO8W.png" /><figcaption>Picture 2: the initial size of the image is 1.16GB.</figcaption></figure><h3>First Optimization: use a light base image</h3><p>In <a href="https://hub.docker.com/">Docker Hub</a> (the public Docker repository) there are several images available for download, each with different characteristics and sizes.</p><p>Normally, images based on <a href="https://hub.docker.com/_/alpine">Alpine</a> or <a href="https://hub.docker.com/_/busybox">BusyBox</a> are extremely small in size compared to those based on other Linux distributions, such as <a href="https://hub.docker.com/_/ubuntu">Ubuntu</a>. This is because Alpine and those other images have been optimized to include the minimum and must-have packages. In the following image you can see a comparison between the sizes of Ubuntu, Alpine, Node and Node based on the alpine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*NyJgka2ebrfG6nzJ.png" /><figcaption>Picture 3: different sizes of base images.</figcaption></figure><p>By modifying the Dockerfile and using Alpine as a base, the final size of our image is 330MB:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/204dbbbb934548984c7f1fe150ffd769/href">https://medium.com/media/204dbbbb934548984c7f1fe150ffd769/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*qwtP4fm0dUL7VnN-.png" /><figcaption>Picture 4: Image size after first optimization is 330MB.</figcaption></figure><h3>Second Optimization: Multi-stage build</h3><p>With multi-stage build, we can use multiple base images in the Dockerfile and copy artifacts, configuration files, etc. from one stage to another, so that we can discard what we don’t need.</p><p>In this example, what we need to deploy the React application is the compiled code; we don’t need the source files, nor the <em>node_modules </em>directory, nor the <em>package.json</em>, etc.</p><p>By changing the Dockerfile to the following one, the final size of our image is 91.5 MB. Keep in mind that the image from the previous stage (lines 1–4) is not automatically deleted, Docker keeps it in cache to run faster if we use the same stage in another build, so it has to be deleted manually .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f033401d411abf31acc769a00ed1dc56/href">https://medium.com/media/f033401d411abf31acc769a00ed1dc56/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*HKFCdIosZROgm8Vx.png" /><figcaption>Picture 5: Image size after second optimization is 91.5MB.</figcaption></figure><p>Now we have a Dockerfile with two stages: in the first one we compile the project and in the second one we deploy the application on the web server. However, a Node container is not the best option to serve web pages (HTML, CSS and JavaScript files, images, etc.), the best option would be to use a server like <a href="https://hub.docker.com/_/nginx">Nginx</a> or <a href="https://hub.docker.com/_/httpd">Apache</a>. In this case I will use Nginx.</p><p>By changing the Dockerfile to the following one, the final size of our image is 22.4 MB. If we run the container, we can see that the web works without problems (<strong><em>Picture 7</em></strong>).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/181fa168ae9a1d9d22538ea716be98b1/href">https://medium.com/media/181fa168ae9a1d9d22538ea716be98b1/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*HkJiAQcGsFz0UXXy.png" /><figcaption>Picture 6: Image size after the third optimization is 22.4MB.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*IX32KFTFcYcEuY5N.png" /><figcaption>Picture 7: result of executing the final container.</figcaption></figure><h4>References</h4><ul><li><a href="https://docs.docker.com/get-started/">https://docs.docker.com/get-started/</a></li><li><a href="https://docs.docker.com/develop/develop-images/multistage-build/">https://docs.docker.com/develop/develop-images/multistage-build/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53fdb4c53311" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-agile-crafter/docker-image-optimization-from-1-16gb-to-22-4mb-53fdb4c53311">Docker Image Optimization: from 1.16GB to 22.4MB</a> was originally published in <a href="https://medium.com/the-agile-crafter">The Agile Crafter’s Notebook (english)</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[17 días ejecutando Google Colab]]></title>
            <link>https://medium.com/the-agile-crafters-notebook-spanish/17-d%C3%ADas-ejecutando-google-colab-158c44080bf?source=rss-57fab778d9e8------2</link>
            <guid isPermaLink="false">https://medium.com/p/158c44080bf</guid>
            <category><![CDATA[google-colab]]></category>
            <category><![CDATA[tips]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Eduardo Ortega]]></dc:creator>
            <pubDate>Sat, 31 Oct 2020 14:08:36 GMT</pubDate>
            <atom:updated>2020-10-31T14:08:36.136Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MYSepwn_bHkb_CdstXvnRg.png" /></figure><p>Para mi proyecto de final de grado en la universidad, necesito ejecutar varios experimentos con diferentes modelos de redes neuronales y descargar archivos comprimidos con los resultados y los modelos aprendidos, para ello estoy usando un entorno conocido como <em>Google Colaboratory</em> o <em>Google Colab</em>. En total, necesito descargar más de 23000 archivos.</p><p>En los últimos 17 días, he ejecutado con éxito 19923 experimentos y me he encontrado con ciertos problemas que me han retrasado, <strong>en este post hablaré sobre esos problemas</strong> para que en el futuro no pasen inadvertidos.</p><h4>1. Definiciones</h4><p><strong><em>Google Colaboratory</em></strong>: es un entorno gratuito de <em>Jupyter Notebook</em> que no requiere configuración y que se ejecuta completamente en la nube. Por otro lado, <em>Jupyter Notebook</em> es un entorno interactivo que permite desarrollar código <em>Python</em> de manera dinámica y se ejecuta como una aplicación cliente-servidor.</p><p><strong><em>Cuaderno</em></strong>: archivos creados en <em>Colab</em> o <em>Jupyter</em>.</p><p><strong><em>Celda</em></strong>: los cuadernos se dividen en pequeños bloques llamados <em>celdas</em>, en ellas se puede ejecutar código <em>Python</em> o renderizar texto usando <em>Markdown</em>.</p><h4>2. Máquinas Virtuales (Sesiones)</h4><p><em>Colab</em> usa máquinas virtuales para ejecutar el código del cuaderno en el servidor, dichas máquinas tienen un tiempo de vida máximo de 12 horas. Por lo tanto, cualquier cosa que tarde más de 12 horas en ejecutarse quedará incompleta.</p><p>Cada <em>cuenta de Google</em> puede tener como máximo 5 sesiones abiertas simultáneamente, en este contexto, una sesión es una máquina virtual. Este límite puede reducirse con el tiempo, por ejemplo: si encadenas varias ejecuciones de 5 sesiones y 12 horas cada una, acabarás teniendo un límite de una única sesión abierta. Este límite dura aproximadamente 12 horas.</p><p>Por este motivo, si necesitas tener 5 sesiones activas siempre, lo mejor es tener una segunda <em>cuenta de Google</em> a la que poder recurrir cuando aparezca la limitación en la primera.</p><h4>3. Conexión a Internet</h4><p>Para que un cuaderno de <em>Colab</em> se ejecute, debes estar conectado a internet. Si tienes mala cobertura WiFi, es posible que la ejecución se pare porque el servidor interpreta que te has desconectado, por eso es mejor que conectes el ordenador al router usando un cable y apagues la tarjeta WiFi del ordenador.</p><h4>4. Mostrar Gráficas</h4><p>Mostrar gráficas en <em>Colab</em> es útil, pero <strong>sólo se mostrarán cuando la ejecución de la celda correspondiente termine</strong>. Si usas una celda que tarda 12 horas en ejecutarse, como yo, no verás las gráficas hasta que pase ese tiempo, o peor aún, es posible que se acumulen tantas gráficas, que la ejecución de la máquina virtual se pare porque has consumido toda la memoria RAM disponible. Además, cuando tienes más de 20 figuras abiertas aparece un <em>warning</em> bastante molesto en la salida de la ejecución.</p><p>Para evitar todo esto, lo mejor es que guardes las gráficas como imagen y luego las cierres, de este modo:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1f59b3257493b141221cea9307e99e4a/href">https://medium.com/media/1f59b3257493b141221cea9307e99e4a/href</a></iframe><h4>5. Evitar la Pérdida de Datos: Conectar con Google Drive</h4><p>Una vez que la máquina virtual se desconecta, lo cual puede pasar por todo lo mencionado anteriormente, se pierden los archivos locales que hubieran en la misma, por eso es buena idea conectar el cuaderno con <em>Google Drive</em> e ir guardando todos los archivos generados en una carpeta. Esto también facilita la descarga de archivos, ya que en <em>Colab</em> no existe la opción de descargar varios archivos a la vez, se tienen que descargar de uno en uno.</p><p>Para conectar con <em>Drive</em>, basta con ejecutar el siguiente código:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c5986804ba9f077b0bcc532fdbf2329a/href">https://medium.com/media/c5986804ba9f077b0bcc532fdbf2329a/href</a></iframe><p>Esto generará un enlace que nos abrirá un cuadro de diálogo, donde seleccionaremos la cuenta de Drive a la que queremos conectarnos y generará una contraseña que se debe introducir en el cuaderno.</p><h4>6. Carga de Archivos desde Google Drive</h4><p>Subir varios archivos a varias sesiones puede ser algo molesto ya que es una tarea repetitiva y que lleva tiempo, dependiendo de la cantidad de archivos y su tamaño. Las máquinas virtuales usan <em>Linux</em>, de modo que, para evitar esta demora, se pueden conectar las sesiones con <em>Google Drive</em>, copiar desde allí los archivos necesarios e importar las funciones después.</p><p>En cuanto a los ficheros de datos, se puede acceder directamente desde <em>Drive</em> sin necesidad de copiarlos, suponiendo que no vayan a sufrir modificaciones durante la ejecución del cuaderno.</p><p>Por ejemplo, si tenemos 3 ficheros fuente, se puede ahorrar tiempo de esta forma:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2e5e6ca9246290f3da55f4086543cad7/href">https://medium.com/media/2e5e6ca9246290f3da55f4086543cad7/href</a></iframe><p>El símbolo “<strong><em>%</em></strong>” significa que vamos a ejecutar un comando de Linux en lugar de código Python.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=158c44080bf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-agile-crafters-notebook-spanish/17-d%C3%ADas-ejecutando-google-colab-158c44080bf">17 días ejecutando Google Colab</a> was originally published in <a href="https://medium.com/the-agile-crafters-notebook-spanish">The Agile Crafter’s Notebook (spanish)</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>