<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Douglas Trajano on Medium]]></title>
        <description><![CDATA[Stories by Douglas Trajano on Medium]]></description>
        <link>https://medium.com/@dougtrajano?source=rss-e513ae1eef97------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*0aKwnNOC9f_zvszJtvgKEQ.jpeg</url>
            <title>Stories by Douglas Trajano on Medium</title>
            <link>https://medium.com/@dougtrajano?source=rss-e513ae1eef97------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:01:18 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dougtrajano/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Introducing pydantic-ai-skills: Composable Agent Skills for the Pydantic AI Ecosystem]]></title>
            <link>https://pub.towardsai.net/introducing-pydantic-ai-skills-composable-agent-skills-for-the-pydantic-ai-ecosystem-dc98dd2bff53?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc98dd2bff53</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Sun, 22 Feb 2026 14:31:00 GMT</pubDate>
            <atom:updated>2026-02-22T14:31:00.613Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Give your AI agents superpowers — without bloating their context window.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*wyAr7FLTk0Q2Ds41AWDImQ.png" /></figure><p>The Agentic AI landscape is evolving fast. We went from simple chatbots to autonomous systems that plan, reason, and execute multi-step workflows. But as agents grow more capable, a familiar engineering problem emerges: <strong>how do you add new capabilities without turning your system prompt into an unmanageable monolith?</strong></p><p>That’s the problem <strong>pydantic-ai-skills</strong> solves. It’s a standardized, composable framework for building and managing Agent Skills within the Pydantic AI ecosystem. Inspired by Anthropic’s Agent Skills open standard — released in December 2025 as a vendor-neutral specification for portable AI capabilities — this library brings the same progressive disclosure pattern to any Pydantic AI agent.</p><h3>The Problem: Context Window Bloat</h3><p>If you’ve built production agents, you know the pain. You start with a clean system prompt, then add instructions for searching the web, then database queries, then report generation, then… suddenly your agent is consuming thousands of tokens before it even reads the user’s message.</p><p>Every new capability means more instructions stuffed into the prompt. Agents get confused. Token costs skyrocket. Maintenance becomes a nightmare.</p><p><strong>pydantic-ai-skills</strong> takes a different approach: <strong>progressive disclosure</strong>. Skills are discovered and loaded on demand, keeping your agent’s context lean and focused. The agent only loads the full instructions for a skill when it actually needs it.</p><h3>How It Works</h3><p>The library implements a tool-based approach that gives agents structured access to skills using 4 tools:</p><ul><li>list_skills()Discover all available skills (names + descriptions).</li><li>load_skill(name)Load complete instructions for a specific skill.</li><li>read_skill_resource(skill_name, resource_name)Access additional documentation, templates, or data files.</li><li>run_skill_script(skill_name, script_name, args)Execute Python scripts defined within a skill.</li></ul><p>This progressive approach means an agent with 50 skills doesn’t need to load all 50 sets of instructions upfront. It sees the menu, picks what it needs, and dives deeper only when required.</p><h3>Getting Started</h3><p>You can quickly install it using the command below.</p><pre>pip install pydantic-ai-skills</pre><h3>Your First Skill</h3><p>Every file-based skill is just a directory with a SKILL.md file. The frontmatter contains metadata, and the markdown body contains instructions for the agent.[1]</p><p>Create a file at ./skills/pydanticai-docs/SKILL.md:</p><pre>---<br>name: pydanticai-docs<br>description: Quick reference for Pydantic AI framework<br>---<br><br># Pydantic AI Docs<br><br>Quick reference for building agents with Pydantic AI.<br><br>## Instructions<br><br>For detailed information, fetch the full docs at:<br>https://ai.pydantic.dev/llms-full.txt<br><br>## Quick Examples<br><br>**Basic Agent:**<br>```python<br>from pydantic_ai import Agent<br><br>agent = Agent(&#39;openai:gpt-4o&#39;)<br>result = agent.run_sync(&#39;Your question&#39;)</pre><pre><br>### Wiring Skills to an Agent<br><br>Now create your agent with the `SkillsToolset`:<br><br>```python<br>import asyncio<br>from pydantic_ai import Agent, RunContext<br>from pydantic_ai_skills import SkillsToolset<br><br>async def main():<br>    # Initialize skills from a directory<br>    skills_toolset = SkillsToolset(directories=[&quot;./skills&quot;])<br><br>    # Create agent with skills as a toolset<br>    agent = Agent(<br>        model=&#39;openai:gpt-4o&#39;,<br>        instructions=&#39;You are a helpful assistant.&#39;,<br>        toolsets=[skills_toolset]<br>    )<br><br>    # Inject skill descriptions into the agent&#39;s instructions<br>    @agent.instructions<br>    async def add_skills(ctx: RunContext) -&gt; str | None:<br>        &quot;&quot;&quot;Add skills instructions to the agent&#39;s context.&quot;&quot;&quot;<br>        return await skills_toolset.get_instructions(ctx)<br><br>    # Run — the agent will discover and use skills automatically<br>    result = await agent.run(<br>        &quot;How do I create a Pydantic AI agent with tools?&quot;<br>    )<br>    print(result.output)<br><br>if __name__ == &quot;__main__&quot;:<br>    asyncio.run(main())</pre><p>That’s it. The agent now automatically discovers the pydanticai-docs skill, loads it when relevant, and uses its instructions to answer questions about Pydantic AI.</p><h3>Script-Based Skills: Going Beyond Instructions</h3><p>Skills aren’t limited to text instructions. You can bundle executable Python scripts that agents can run on demand. Here’s a calculator skill with a script:</p><pre>./skills/<br>└── calculator/<br>    ├── SKILL.md<br>    └── scripts/<br>        └── calculate.py</pre><p><strong>SKILL.md:</strong></p><pre>---<br>name: calculator<br>description: Perform calculations using Python<br>---<br><br># Calculator Skill<br><br>Use the `calculate` script to perform mathematical operations.</pre><p><strong>scripts/calculate.py:</strong></p><pre>&quot;&quot;&quot;Evaluate a Python expression and return the result as JSON.&quot;&quot;&quot;<br><br>import argparse<br>import json<br>import sys<br><br>parser = argparse.ArgumentParser()<br>parser.add_argument(&#39;--expression&#39;, required=True, help=&#39;Python expression to evaluate&#39;)<br>args = parser.parse_args()<br><br>try:<br>    result = eval(args.expression)<br>    print(json.dumps({&#39;result&#39;: result}))<br>except Exception as e:<br>    print(f&#39;Error: {e}&#39;, file=sys.stderr)<br>    sys.exit(1)</pre><p>When the agent encounters a math problem, it can call run_skill_script(skill_name=&quot;calculator&quot;, script_name=&quot;calculate&quot;, args={&quot;expression&quot;: &quot;2**10&quot;}) and get {&quot;result&quot;: 1024} back.</p><h3>Programmatic Skills: Dynamic Capabilities in Code</h3><p>File-based skills are great for stable, shareable capabilities. But sometimes you need skills that are generated at runtime, depend on live configuration, or require dependency injection. That’s where <strong>programmatic skills</strong> come in.</p><pre>from pydantic_ai import Agent, RunContext<br>from pydantic_ai_skills import Skill, SkillsToolset, SkillResource<br><br># Create a skill object directly in Python<br>my_skill = Skill(<br>    name=&#39;data-processor&#39;,<br>    description=&#39;Process and analyze data&#39;,<br>    content=&#39;Use this skill for data analysis tasks.&#39;,<br>    resources=[<br>        SkillResource(<br>            name=&#39;reference&#39;,<br>            content=&#39;## Reference\n\nStatic reference documentation here...&#39;<br>        )<br>    ]<br>)<br><br># Add a dynamic resource via decorator<br>@my_skill.resource<br>def get_schema() -&gt; str:<br>    &quot;&quot;&quot;Get current data schema.&quot;&quot;&quot;<br>    return &quot;Available fields: id, name, value, timestamp&quot;<br><br># Add an executable script via decorator<br>@my_skill.script<br>async def process_data(ctx: RunContext, query: str) -&gt; str:<br>    &quot;&quot;&quot;Process data based on query.&quot;&quot;&quot;<br>    results = await ctx.deps.process(query)<br>    return f&quot;Processed {len(results)} records&quot;<br><br># Wire it up<br>skills_toolset = SkillsToolset(skills=[my_skill])</pre><p>Programmatic skills support full RunContext dependency injection, allowing your scripts and resources to access databases, APIs, and any other runtime dependencies your agent needs.</p><h3>The Decorator Pattern: Inline Skill Definitions</h3><p>For skills tightly coupled with agent initialization, the @toolset.skill() decorator offers the most concise syntax:</p><pre>from pydantic_ai import Agent, RunContext<br>from pydantic_ai_skills import SkillsToolset<br><br>skills_toolset = SkillsToolset()<br><br>@skills_toolset.skill()<br>def data_analyzer() -&gt; str:<br>    &quot;&quot;&quot;Analyze data from various sources.&quot;&quot;&quot;<br>    return &quot;&quot;&quot;<br>    # Data Analysis Skill<br><br>    Use this skill to analyze datasets and generate insights.<br>    &quot;&quot;&quot;<br><br>@data_analyzer.resource<br>async def get_schema(ctx: RunContext) -&gt; str:<br>    &quot;&quot;&quot;Get database schema from live connection.&quot;&quot;&quot;<br>    schema = await ctx.deps.database.get_schema()<br>    return f&quot;## Current Schema\n{schema}&quot;<br><br>agent = Agent(<br>    model=&#39;openai:gpt-4o&#39;,<br>    toolsets=[skills_toolset]<br>)<br><br>@agent.instructions<br>async def add_skills(ctx: RunContext) -&gt; str | None:<br>    &quot;&quot;&quot;Add skills instructions to the agent&#39;s context.&quot;&quot;&quot;<br>    return await skills_toolset.get_instructions(ctx)</pre><h3>Mixing File-Based and Programmatic Skills</h3><p>One of the most powerful patterns is combining both approaches — stable, reusable skills from files alongside dynamic, runtime-specific skills defined in code:</p><pre>from pydantic_ai_skills import SkillsToolset<br><br># Load file-based skills AND add programmatic ones<br>skills_toolset = SkillsToolset(<br>    directories=[<br>        &quot;./skills/research&quot;,     # Shared research skills<br>        &quot;./skills/data&quot;,         # Data processing skills<br>    ],<br>    validate=True,<br>)<br><br># Add a runtime-specific skill inline<br>@skills_toolset.skill(name=&#39;runtime-monitor&#39;, metadata={&#39;version&#39;: &#39;1.0.0&#39;})<br>def monitoring() -&gt; str:<br>    return &quot;Monitor application runtime metrics&quot;</pre><h3>agentskills.io Compatibility</h3><p>The library implements the <a href="https://agentskills.io/home">agentskills.io</a> specification, which was released by Anthropic as an open standard in December 2025 and adopted by Microsoft, OpenAI, Atlassian, Figma, Cursor, and GitHub. Skills created with pydantic-ai-skills are structurally compatible with Anthropic&#39;s skill format — including the SKILL.md file structure, naming conventions (lowercase, hyphens, ≤64 chars), and the progressive disclosure loading pattern.</p><p>This means skills built for this library can potentially be reused across different platforms that support the Agent Skills standard.</p><h3>Security Considerations</h3><p>Since skills provide agents with new capabilities through instructions and executable code, security is critical. The library implements several safeguards:</p><ol><li><strong>Path traversal prevention</strong> — Scripts and resources must be within the skill directory</li><li><strong>Script timeout</strong> — Scripts are killed after a configurable timeout</li><li><strong>Safe subprocess execution</strong> — Scripts run in subprocesses with limited privileges</li><li><strong>Validation</strong> — Automatic validation of skill metadata and structure</li></ol><p>The documentation strongly recommends using only skills from trusted sources — those created in-house or obtained from vetted providers.[1]</p><h3>Why pydantic-ai-skills?</h3><p>The library was inspired by the pydantic-deepagents project, which pioneered many of the skill patterns in the Pydantic AI ecosystem. Here&#39;s what sets pydantic-ai-skills apart:</p><ol><li><strong>Focused scope</strong> — It does one thing well: skill management. No planning loops, no filesystem toolsets, no subagent delegation. Just clean, composable skills.</li><li><strong>Drop-in integration</strong> — Works as a standard Pydantic AI Toolset, so it plugs into any existing agent without architectural changes.</li><li><strong>Type-safe</strong> — Built on Python dataclasses and type hints with full type annotation support.</li><li><strong>Agentskills.io spec</strong> — Structurally aligned with the Agent Skills open standard.</li><li><strong>Lightweight</strong> — Minimal dependencies; install and use in minutes.</li></ol><h3>Real-World Example: Research Assistant</h3><p>Here’s a complete example of a research assistant that uses an ArXiv search skill:</p><pre>import asyncio<br>from pydantic_ai import Agent, RunContext<br>from pydantic_ai_skills import SkillsToolset<br><br>async def main():<br>    skills_toolset = SkillsToolset(directories=[&quot;./examples/skills&quot;])<br><br>    agent = Agent(<br>        model=&#39;openai:gpt-4o&#39;,<br>        instructions=(<br>            &quot;You are a research assistant specializing in &quot;<br>            &quot;academic papers.&quot;<br>        ),<br>        toolsets=[skills_toolset]<br>    )<br><br>    @agent.instructions<br>    async def add_skills(ctx: RunContext) -&gt; str | None:<br>        return await skills_toolset.get_instructions(ctx)<br><br>    result = await agent.run(<br>        &quot;Find the 5 most recent papers on transformer architectures&quot;<br>    )<br>    print(result.output)<br><br>if __name__ == &quot;__main__&quot;:<br>    asyncio.run(main())</pre><p>The agent discovers the arxiv-search skill, loads its instructions, and uses run_skill_script to execute the search — all without any ArXiv-specific code in the agent itself.</p><h3>What’s Next</h3><p>The library is at version 0.4.1 and actively maintained. The documentation includes guides on creating skills, programmatic skills, advanced features like custom script executors and dependency injection, implementation patterns, and security best practices.</p><p>See more on:</p><ul><li><strong>Documentation</strong>: <a href="https://dougtrajano.github.io/pydantic-ai-skills/">https://dougtrajano.github.io/pydantic-ai-skills/</a></li><li><strong>GitHub</strong>: <a href="https://github.com/DougTrajano/pydantic-ai-skills">https://github.com/DougTrajano/pydantic-ai-skills</a></li><li><strong>PyPI</strong>: <a href="https://pypi.org/project/pydantic-ai-skills/">https://pypi.org/project/pydantic-ai-skills/</a></li></ul><p>If you’re building Pydantic AI agents and want to keep them modular, maintainable, and efficient, give pydantic-ai-skills a try. Skills are the building blocks of capable agents, and progressive disclosure is how you scale without drowning in context.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc98dd2bff53" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/introducing-pydantic-ai-skills-composable-agent-skills-for-the-pydantic-ai-ecosystem-dc98dd2bff53">Introducing pydantic-ai-skills: Composable Agent Skills for the Pydantic AI Ecosystem</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying MLflow in AWS App Runner]]></title>
            <link>https://medium.com/mlops-community/deploying-mlflow-in-aws-app-runner-cc6caf7fb8e3?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/cc6caf7fb8e3</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[mlflow]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Mon, 26 Jul 2021 14:10:05 GMT</pubDate>
            <atom:updated>2022-07-02T22:31:39.785Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*pZEfMJWixM88kOdJ-P0jUQ.png" /><figcaption><a href="https://www.mlflow.org/">mlflow.org</a></figcaption></figure><p><a href="http://aws.amazon.com/">AWS</a> launched May 2021 the <a href="https://aws.amazon.com/apprunner/">AWS App Runner</a>, a fully managed service that makes it easy for developers to quickly deploy containerized web applications and APIs. [1]</p><p>Unlike <a href="https://aws.amazon.com/fargate/">AWS Fargate</a>, <a href="https://aws.amazon.com/apprunner/">AWS App Runner</a> has a pricing policy based on requests that your application receives. Apps that don’t receive requests all the time can benefit from it.</p><p>Our loved <a href="https://www.mlflow.org/">MLflow</a> fits perfectly in this context. The interaction with <a href="https://www.mlflow.org/">MLflow</a> is usually made in training jobs, data scientists evaluating their experiments, or APIs that expose our models. In all these cases we don’t have frequent access, it occurs eventually.</p><p>So, in this post, I’ll describe how I deployed the MLflow with basic auth (username/password) in AWS App Runner for my personal projects and the costs involved in this solution.</p><h3>Solution Overview</h3><p>The following diagram illustrates this architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*aEUEba-IIq_hm1gyidNFEg.png" /><figcaption>Figure 1: Architecture. Image by author</figcaption></figure><p>An MLflow tracking server also has two components to store data: a backend store and an artifact store. [3]</p><p>The <strong>backend store</strong> is where MLflow Tracking Server stores experiment and run metadata as well as params, metrics, and tags for runs. [3]</p><p>I used the <a href="https://aws.amazon.com/rds/mysql/">Amazon RDS</a> as our backend store because it has a Free Tier available for 12 months. Each calendar month, the free tier will allow you to use the Amazon RDS resources listed below for free: [4]</p><ul><li>750 hrs of Amazon RDS in a Single-AZ “db.t2.micro” Instance.</li><li>20 GB of General Purpose Storage (SSD).</li><li>20 GB for automated backup storage and any user-initiated DB Snapshots.</li></ul><p>The <strong>artifact store</strong> is a location suitable for large data and is where data scientists log their artifact output (for example, models). [3] Obviously, the default choice here is an S3 Bucket.</p><p>Amazon S3 is an object storage service. It also has a free tier available for 12 months. [5]</p><ul><li>5GB of standard storage.</li><li>20,000 Get Requests</li><li>2,000 Put Requests</li></ul><p>Our MLflow Tracking Server will be hosted in App Runner. This new service launched by AWS is very similar to <a href="https://cloud.google.com/run">Cloud Run</a> which is available in <a href="https://cloud.google.com/">Google Cloud</a>. With App Runner, you deploy and run your applications in container instances, which consume compute and memory resources.</p><p>When your application is idle, you pay per GB of memory for provisioned container instances that keep your application warm and eliminate cold starts. [2]</p><p>When requests come in, your application responds in milliseconds, and you pay for the vCPU and memory consumed by your active container instances as your application is processing requests. [2] See more about the pricing <a href="https://aws.amazon.com/apprunner/pricing/">here</a>.</p><h3>Deployment</h3><p>I created a Terraform stack that will help you to easily create all resources needed to host the MLflow Server with basic authentication.</p><p>The Terraform stack will create the following resources:</p><ul><li>An <a href="https://aws.amazon.com/s3/">S3 bucket</a> for storing MLflow artifacts.</li><li>An <a href="https://aws.amazon.com/iam/">IAM role and policy</a> for the MLflow server connect to the S3 bucket.</li><li>An <a href="https://aws.amazon.com/rds/">RDS</a> database instance (MySQL) for storing MLflow metadata.</li><li>A service in <a href="https://aws.amazon.com/apprunner/">App Runner</a> to run MLflow Tracking Server.</li></ul><h4>Prerequisites</h4><p>You’ll need to have the following installed:</p><ul><li><a href="https://aws.amazon.com/cli/">AWS CLI</a> (1.18.69)</li><li><a href="https://www.terraform.io/downloads.html">Terraform CLI</a> (v1.0.1)</li></ul><h4>Deploying MLflow</h4><ol><li><a href="https://aws.amazon.com/free/">Create an AWS account</a> if you don’t already have one.</li><li>Configure AWS CLI to use your AWS account.</li><li>Clone the repository: <a href="https://github.com/DougTrajano/mlflow-server/">github.com/DougTrajano/mlflow-server</a></li><li>Open mlflow-server/terraform folder.</li></ol><pre>cd mlflow-server/terraform</pre><p>5. Run the following command to create all the required resources:</p><pre>terraform apply -var mlflow_username=&quot;USERNAME&quot; -var mlflow_password=&quot;PASSWORD&quot;</pre><p>5. Type “yes” when prompted to continue.</p><blockquote><strong>mlflow_username </strong>and <strong>mlflow_password </strong>are used to authenticate you in MLflow UI and API. These variables are informed to <a href="https://www.nginx.com/">Nginx</a> proxy as environment variables in docker. If you don’t provide a mlflow_password, this stack will create a strong password for you. :)</blockquote><p>Terraform will create the resources in your account, this may take few minutes. In the end, you will receive the MLflow URL (<strong>XXXXX.aws-region.awsapprunner.com</strong>) and username/password for basic authentication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*L8yuhLGvH8LhpEjqvTzYzA.png" /><figcaption>Figure 2: Terraform execution. Image by author</figcaption></figure><p>Access the service_url provided and see your MLflow \o/</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dmg7LxO7JRLkWOQqtL2G-g.png" /><figcaption>Figure 3: MLflow UI. Image by author</figcaption></figure><p>You can use this link to access the MLflow UI and to track experiments in MLflow API, for example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f3ea8941d9eacf0a2d9811fa7683586b/href">https://medium.com/media/f3ea8941d9eacf0a2d9811fa7683586b/href</a></iframe><p>So, that’s it! If you want to see the code and more information about this project, see the repository below.</p><p><a href="https://github.com/DougTrajano/mlflow-server">DougTrajano/mlflow-server</a></p><p>In this solution, Amazon RDS and Amazon S3 are both parts of the free tier provided by AWS. The only cost involved here is App Runner, but based on my experience and AWS estimates, its cost will be close to 10 dollars a month, depending on your usage.</p><p>This post does not cover MLflow usage, but if you want to see how to use MLflow like a pro, I strongly recommend the post below. :)</p><p><a href="https://towardsdatascience.com/complete-data-science-project-template-with-mlflow-for-non-dummies-d082165559eb">Complete Data Science Project Template with Mlflow for Non-Dummies.</a></p><p>I hope you enjoyed this post. If you have any comments or contributions, leave a comment and make the community stronger :)</p><h3>References</h3><p>[1]”AWS App Runner — Fully managed container application service — Amazon Web Services”, <em>Amazon Web Services, Inc.</em>, 2021. [Online]. Available: <a href="https://aws.amazon.com/apprunner/">https://aws.amazon.com/apprunner/.</a> [Accessed: 17- Jul- 2021]</p><p>[2]”AWS App Runner Pricing — Fully managed container application service — Amazon Web Services”, <em>Amazon Web Services, Inc.</em>, 2021. [Online]. Available: <a href="https://aws.amazon.com/apprunner/pricing/">https://aws.amazon.com/apprunner/pricing/.</a> [Accessed: 17- Jul- 2021]</p><p>[3]”MLflow Tracking — MLflow 1.19.0 documentation”, <em>Mlflow.org</em>, 2021. [Online]. Available: <a href="https://www.mlflow.org/docs/latest/tracking.html.">https://www.mlflow.org/docs/latest/tracking.html.</a> [Accessed: 17- Jul- 2021]</p><p>[4]”Amazon RDS | Cloud Relational Database | Amazon Web Services”, <em>Amazon Web Services, Inc.</em>, 2021. [Online]. Available: <a href="https://aws.amazon.com/rds/">https://aws.amazon.com/rds/.</a> [Accessed: 17- Jul- 2021]</p><p>[5]”Cloud Object Storage | Store &amp; Retrieve Data Anywhere | Amazon Simple Storage Service (S3)”, <em>Amazon Web Services, Inc.</em>, 2021. [Online]. Available: <a href="https://aws.amazon.com/s3/">https://aws.amazon.com/s3/.</a> [Accessed: 17- Jul- 2021]</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cc6caf7fb8e3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/mlops-community/deploying-mlflow-in-aws-app-runner-cc6caf7fb8e3">Deploying MLflow in AWS App Runner</a> was originally published in <a href="https://medium.com/mlops-community">MLOps.community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Otimizando seu chatbot com a Anallyticabot]]></title>
            <link>https://medium.com/data-hackers/otimizando-seu-chatbot-com-a-anallyticabot-9e76e3d864f7?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/9e76e3d864f7</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ibm]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Thu, 10 Dec 2020 04:39:36 GMT</pubDate>
            <atom:updated>2020-12-10T04:39:36.325Z</atom:updated>
            <content:encoded><![CDATA[<p>Conheça a Anallyticabot! Uma poderosa aplicação open source que te ajuda a desenvolver e melhorar seu chatbot IBM Watson Assistant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*KX8_KE1JWjUD1XDKmb8igA.gif" /><figcaption>Apresentando a Anallyticabot! \o/</figcaption></figure><p>A Anallyticabot é uma aplicação web que aplica técnicas de <strong>data science</strong> e <strong>advanced analytics</strong> para ajudar você no desenvolvimento e melhoria do seu chatbot.</p><p>Um projeto pessoal meu que tem como objetivo retribuir o tanto que eu aprendi com a comunidade open source! ❤</p><p><a href="https://github.com/DougTrajano/anallyticabot">DougTrajano/anallyticabot</a></p><p>Ela foi desenvolvida com base no <a href="https://www.ibm.com/cloud/watson-assistant">IBM Watson Assistant</a>, uma solução de assistente virtual baseado em IA criada pela <a href="https://cloud.ibm.com/">IBM Cloud</a>.</p><p>A Anallyticabot foi desenvolvida com o <a href="https://www.streamlit.io/">Streamlit</a>, uma maneira simples e fácil de criar Data Apps no Python.</p><p>Vamos experimentar? Neste post você vai ver como preparar a Anallyticabot para rodar na sua máquina.</p><h3>Instalando a Anallyticabot</h3><p>A Anallyticabot pode ser utilizada de duas formas diferentes (Docker ou localmente).</p><p>Aqui iremos abordar as duas formas para que você escolha a opção que preferir. :)</p><h4>Docker (recomendado)</h4><p>Você precisará ter o Docker instalado na sua máquina.</p><p>Não sabe o que é Docker? Veja este artigo <a href="https://medium.com/devs-javagirl/docker-criando-o-seu-primeiro-cont%C3%AAiner-61ac022ceeef">aqui</a> feito pela <a href="https://medium.com/@lauradamaceno">Laura Damaceno</a> para a <a href="https://medium.com/devs-javagirl">Devs JavaGirl</a>.</p><p>Com o Docker instalado, execute o comando abaixo para baixar e executar a imagem da Anallyticabot:</p><pre>docker run --name anallyticabot dougtrajano/anallyticabot </pre><p>O comando acima irá baixar e executar a imagem Docker da Anallyticabot.</p><p>Ao final da execução, uma mensagem com as URLs de acesso deverá aparecer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*Zb3tQ2TxosF_CfcvrDG8BQ.png" /><figcaption>URLs para acessar a aplicação localmente</figcaption></figure><h4>Clonando o repositório</h4><p>Para instalar localmente, você precisa ter Python 3.6.x ou mais instalado.</p><p>Execute os seguintes comandos abaixo na pasta que você deseja instalar a aplicação.</p><p>O primeiro comando é para clonar o repositório localmente:</p><pre>git clone https://github.com/DougTrajano/anallyticabot.git </pre><p>Depois, você precisará abrir a pasta do repositório que acabou de ser clonado:</p><pre>cd anallyticabot</pre><p>Então, devemos instalar as libs necessárias para a aplicação:</p><pre>pip install -r requirements.txt</pre><p>Por último, basta executar o comando abaixo para então hostear sua própria Anallyticabot:</p><pre>streamlit run anallyticabot.py</pre><p>Ao executar o comando acima, ele irá exibir no console as URLs de acesso a aplicação:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*Zb3tQ2TxosF_CfcvrDG8BQ.png" /><figcaption>URLs para acessar a aplicação localmente</figcaption></figure><p>Acesse uma das URLs abaixo em seu navegador para acessar a Anallyticabot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zVd2Hb2-Jl_Pmgs3trse5w.png" /><figcaption>Anallyticabot — Home</figcaption></figure><h3>Conectando ao Watson Assistant</h3><p>Como a Anallyticabot foi pensada em assistentes virtuais que usam a engine do IBM Watson Assistant, você precisará conectar uma skill do Watson Assistant na aba de “Settings” para utilizar as análises.</p><p>Quer saber como desenvolver seu chatbot no IBM Watson Assistant? Este artigo <a href="https://medium.com/ibmdeveloperbr/watson-assistant-como-criar-o-seu-chatbot-usando-skills-e-assistants-755b4677984b">aqui</a> feito pelo <a href="https://medium.com/@victorshinya">Victor Shinya</a> para a <a href="https://medium.com/ibmdeveloperbr">IBM Developer Brasil</a> vai te ajudar com isto.</p><p>Bom, vamos ver agora como conectar a Anallyticabot com sua skill do IBM Watson Assistant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z7R4N42akKVCnOYDwQ7bXQ.png" /><figcaption>Anallyticabot - Settings</figcaption></figure><p>Na tela de Settings, preencha os campos disponíveis na sessão “Watson Assistant - Credentials”</p><p>O <strong>Chatbot name</strong> é opcional, não precisa ser preenchido.</p><p>Preencha a <strong>Skill ID</strong> e a <strong>API key</strong> da sua instância. Este artigo <a href="https://cloud.ibm.com/apidocs/assistant/assistant-v2#finding-credentials-in-the-ui">aqui</a> na documentação da IBM Cloud pode te ajudar com isto.</p><p>Revise se o <strong>Region</strong> é o mesmo que você utiliza no seu Watson Assistant, por default, é utilizado a região de Dallas.</p><p>Feito isto, basta clicar em <strong>“Connect”</strong>.</p><p>Com sua skill do Watson Assistant conectado, o menu lateral apresentará as páginas que levam as análises</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/322/1*s-xbYR3VtLoWGDcU00i5NA.png" /><figcaption>Anallyticabot - Menu</figcaption></figure><p>Cada página possui uma análise específica que você poderá executar.</p><p>Para explicar melhor como você pode aproveitar cada uma dessas análises, estou preparando uma série de artigos que falarei no detalhe sobre o uso de cada uma delas.</p><p>Chegamos ao fim deste post que tinha como objetivo apresentar para você a Anallyticabot e como prepará-la em sua máquina local.</p><p>Tem uma sugestão para a Anallyticabot? Comenta ai que vamos entender essa necessidade. :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9e76e3d864f7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-hackers/otimizando-seu-chatbot-com-a-anallyticabot-9e76e3d864f7">Otimizando seu chatbot com a Anallyticabot</a> was originally published in <a href="https://medium.com/data-hackers">Data Hackers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Coletando bananas com Deep Reinforcement Learning]]></title>
            <link>https://medium.com/data-hackers/coletando-bananas-com-deep-reinforcement-learning-d174cf0cf006?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/d174cf0cf006</guid>
            <category><![CDATA[inteligencia-artificial]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Sun, 02 Aug 2020 20:57:16 GMT</pubDate>
            <atom:updated>2020-08-03T02:21:20.823Z</atom:updated>
            <content:encoded><![CDATA[<p>Que negócio é esse de Deep Reinforcement Learning? Como funciona? Do que se alimenta? Veja na sexta, no Globo Reporter (brinks)</p><p>Neste post vamos ver um agente que utiliza um algoritmo de Deep Reinforcement Learning para aprender quais as bananas ele deve coletar (amarelas) e quais ele não deve coletar (roxas).</p><p>Antes de falarmos sobre o algoritmo e o agente, vamos entender como reinforcement learning funciona, ok?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*elowxw4Ts2OZ4FLV.jpg" /><figcaption>Você esperando eu falar rapidinho sobre reinforcement learning</figcaption></figure><h3>Bem, como reinforcement learning funciona?</h3><p>Reinforcement learning é uma subclasse de Machine Learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/0*rrTUmRVpFR4LRIab.jpg" /><figcaption>Tipos de aprendizado de máquina</figcaption></figure><p>Falando de Reinforcement learning, teremos alguns conceitos importantes para entender:</p><p><strong>Agente (agent):</strong> É o nosso modelo que deverá interagir com o environment (ambiente), ele irá receber os <strong>estados</strong> (states) do environment e fornecer uma <strong>ação</strong> (action) que deseja realizar.</p><p><strong>Ambiente (environment):</strong> É o local onde o agente estará inserido. O environment irá fornecer uma <strong>recompensa</strong> (reward) para o agente em toda <strong>ação</strong> que o agente fornecer.</p><p>A imagem abaixo ilustra bem esse pensamento.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AqZCU_45QkPdQr-N.png" /></figure><p>Os algoritmos de reinforcement learning possuem técnicas diferentes para alcançar seu objetivo, mas no geral, todos possuem o mesmo objetivo: <strong>maximizar a recompensa cumulativa</strong>.</p><p>Basicamente eles procurarão alcançar o maior valor possível antes do ambiente acabar (sim, ele acaba).</p><p>Alguns que são baseados em encontrar a melhor <strong>value function</strong> (função valor), outros são baseados em encontrar a política que reflete a melhor value function.</p><p>Para entender melhor a diferença entre algoritmos value-based e policy-based. Vamos ver um exemplo prático:</p><blockquote>Você gerencia um estoque de produtos e decidiu que quando a contagem de produtos cai abaixo de um determinado valor, você emite um pedido de compra para repor seu estoque. Essa política é mais simples do que entender o comportamento e preferências dos seus clientes para prever o impacto no estoque.</blockquote><p>Por “simples”, entenda que temos algoritmos que funcionarão melhor em cada ambiente.</p><p>Temos ambientes que funcionarão melhor com algoritmos baseados em política, outros, com algoritmos baseados em função de valor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*QcDyxUnj6KuYvL3oa0QjLw.png" /><figcaption>Exemplos de algoritmos de Reinforcement Learning</figcaption></figure><p>Tá, até agora a gente estava falando de reinforcement learning no geral. Afinal, o que e “Deep Reinforcement Learning”?</p><p>Basicamente, <strong>Deep Reinforcement Learning</strong> é a combinação de <strong>Deep Learning </strong>com <strong>Reinforcement Learning.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jtjiyePjrtEv47ks" /><figcaption>Reinforcement Learning + Deep Learning = Deep Reinforcement Learning</figcaption></figure><p>Bom, para o ambiente que usei neste projeto, usei o algoritmo Deep Q-Learning. Basicamente ele possui uma neural network (rede neural) que será responsável por encontrar a melhor value function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*maV3KttDSSao8aKc.png" /><figcaption>Deep Q-Learning</figcaption></figure><p>Bom, agora que você sabe um pouco mais sobre Reinforcement learning e Deep Reinforcement learning, vamos falar de bananas!</p><h3>Bananas collector</h3><p>Este ambiente é fornecido pela <a href="https://unity3d.com/machine-learning/">Unity</a>.</p><p>Unity é uma das empresas mais avançadas nesta área, vale a pena dar uma olhada no site deles.</p><p>Bom, para entender o ambiente em que vamos atuar, veja o vídeo abaixo:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FheVMs3t9qSk%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DheVMs3t9qSk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FheVMs3t9qSk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/40043c084a871d48ea3bce772a44c2c4/href">https://medium.com/media/40043c084a871d48ea3bce772a44c2c4/href</a></iframe><h4>Estados</h4><p>Os estados do ambiente terão <strong>37</strong> elementos que informarão a velocidade do agente, juntamente com a percepção baseada em raios de objetos em torno da direção direta do agente.</p><h4>Ações</h4><p>Nós temos <strong>4</strong> ações disponíveis que o agente poderá realizar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/0*1OaCIwac3Zp5tjxG.png" /></figure><h4>Recompensa</h4><p>Uma recompensa de <strong>+1</strong> é fornecida para cada banana amarela que o agente coleta e uma recompensa de <strong>-1</strong> é fornecida para cada banana azul.</p><h3>Arquitetura do agente</h3><p>O agente criado nesse projeto possui um modelo baseado no algoritmo Deep Q-Learning e uma memória que armazena os estados, ações, recompensas, etc.</p><h4>Agente</h4><p>O agente é uma classe que concentra tudo que precisamos para interagir com o environment. Ele possui os seguintes métodos: <strong>step(), act(), learn()</strong> entre outros.</p><p>O modelo e a memória fazem parte do agente (como atributos).</p><h4>Deep Q-Learning</h4><p>O propósito deste modelo é entender os estados e fornecer ações que o agente deverá fornecer. Basicamente ele possui uma input layer (camada de entrada), uma hidden layer (camada oculta) e uma output layer (camada de saída).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7668eb8d4bbcbc3bb513a273181ef07e/href">https://medium.com/media/7668eb8d4bbcbc3bb513a273181ef07e/href</a></iframe><h4>Memória</h4><p>Temos duas implementações de memória disponíveis:</p><p><strong>ReplayMemory</strong></p><p>É uma implementação mais simples. Basicamente armazena os estados, ações e recompensas, depois fornece aleatoriamente estes conjuntos para treinar o modelo.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ff40d1720449d579c7b6749b189ef465/href">https://medium.com/media/ff40d1720449d579c7b6749b189ef465/href</a></iframe><p><strong>PrioritizedMemory</strong></p><p>Essa implementação é um pouquinho mais complexa, pois as experiências para treinamento não serão fornecidas aleatoriamente. Aqui iremos tentar entender quais experiências o agente provavelmente irá aprender mais, usando weights (pesos) para cada experiência que o agente teve.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b350869cb922b1b0dbe368c4c3581816/href">https://medium.com/media/b350869cb922b1b0dbe368c4c3581816/href</a></iframe><h3>Treinando o agente</h3><p>Para resolver este ambiente é necessário alcançar um score médio maior que 13+.</p><p>O agente possui vários hiperparâmetros que são usados pelo modelo, pela memória, etc.</p><p>agent = Agent(state_size=<strong>37</strong>, action_size=<strong>4</strong>, seed=<strong>199</strong>, nb_hidden=<strong>(64, 64)</strong>, learning_rate=<strong>0.001</strong>, memory_size=<strong>int(1e6)</strong>, prioritized_memory=<strong>False</strong>, batch_size=<strong>64</strong>, gamma=<strong>0.9</strong>, tau=<strong>0.03</strong>, small_eps=<strong>0.03</strong>, update_every=<strong>4</strong>)</p><p>No treinamento, testamos dois agentes, um com a ReplayMemory e outro com a PrioritizedMemory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/0*Jf_sC3_ZKmw5BKcW.png" /><figcaption>Agente com ReplayMemory</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/475/0*RLWyFY5gKlKFO9vB.png" /><figcaption>Agente com PrioritizedMemory</figcaption></figure><p>O agente com a PrioritizedMemory aprendeu com menos episódios do que o agente com a ReplayMemory, porém, levou 11h30 para isto.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*MK0cJkMB3jJDEm_8.gif" /><figcaption>Testando o agente com PrioritizedMemory</figcaption></figure><p>Você pode ver todo o código no meu repositório do GitHub</p><p><a href="https://github.com/DougTrajano/ds_drl_bananas_collector">DougTrajano/ds_drl_bananas_collector</a></p><h3>Considerações</h3><p>Estou estudando bastante sobre Deep Reinforcement Learning nos últimos meses, principalmente motivado pela proximidade destes algoritmos com a forma em que agimos em determinadas circunstâncias.</p><p>O grande desafio para usar estes algoritmos em nosso dia a dia é ter o ambiente desenvolvido, mapeando os estados que o agente receberá conforme suas ações. Afinal, nós usamos um ambiente pronto aqui, no mundo real precisaremos também desenvolver o environment.</p><p>Como trabalho futuro, é possível implementar uma arquitetura dueling DQN no modelo deste agente. Veja mais sobre isto <a href="https://arxiv.org/abs/1511.06581">aqui</a>.</p><p>Obrigado por acompanhar até aqui e qualquer comentário será muito bem-vindo!</p><p>Ah! Gostaria de colocar alguns outros artigos muito legais sobre Deep Reinforcement Learning:</p><ul><li><a href="https://paulovasconcellos.com.br/explicando-deep-reinforcement-learning-com-super-mario-ao-inv%C3%A9s-de-matem%C3%A1tica-4c77392cc733">Explicando Deep Reinforcement Learning com Super Mario ao invés de matemática</a> por Paulo Vasconcellos</li><li><a href="https://medium.com/data-hackers/deep-reinforcement-learning-aprendendo-a-jogar-atari-99c79becbf19">Deep Reinforcement Learning — Aprendendo a jogar Atari</a> por Lucas Oliveira em Data Hackers</li><li><a href="https://medium.com/data-hackers/multi-armed-bandits-como-fazer-boas-escolhas-e73a8422b71d">Multi-Armed Bandits: Como fazer boas escolhas</a> por Marlesson Santana em Data Hackers</li><li><a href="https://medium.com/analytics-vidhya/collecting-bananas-with-drl-agent-bdd8bf216d11">Collecting bananas with Deep Reinforcement Learning agent</a> (inglês)</li><li><a href="https://towardsdatascience.com/policy-based-reinforcement-learning-the-easy-way-8de9a3356083">Policy Based Reinforcement Learning, the Easy Way</a> (inglẽs)</li></ul><h3>Referências</h3><ul><li><a href="https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893">Deep Reinforcement Learning Nanodegree - Udacity</a></li><li><a href="https://arxiv.org/pdf/1511.05952.pdf">Prioritized Experience Replay - Google DeepMind</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d174cf0cf006" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-hackers/coletando-bananas-com-deep-reinforcement-learning-d174cf0cf006">Coletando bananas com Deep Reinforcement Learning</a> was originally published in <a href="https://medium.com/data-hackers">Data Hackers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Training a double-jointed arm with DDPG — Deep Reinforcement Learning]]></title>
            <link>https://medium.com/analytics-vidhya/training-a-double-jointed-arm-with-ddpg-deep-reinforcement-learning-e25529a5915b?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/e25529a5915b</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[deep-reinforcement]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Sat, 18 Jul 2020 13:51:37 GMT</pubDate>
            <atom:updated>2020-07-18T16:03:04.892Z</atom:updated>
            <content:encoded><![CDATA[<h3>Training a double-jointed arm with DDPG — Deep Reinforcement Learning</h3><p>Today we’ll see another powerful Deep Reinforcement Learning algorithm.</p><p>It’s important that you can know the basics about the Deep Reinforcement Learning, you can see my last article <a href="https://medium.com/analytics-vidhya/collecting-bananas-with-drl-agent-bdd8bf216d11?source=friends_link&amp;sk=c1c829667c90e9a535af585864f083f2">here</a> (Collecting bananas with Deep Reinforcement Learning).</p><p>Before we start, I’ll share with you something more about Deep Reinforcement Learning. It’s fast, I promise!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*-Jp_qGo-BBmDs_jU.gif" /></figure><h3>Theory first</h3><p>Reinforcement Learning is a subclass from Artificial Intelligence, and this subclass has different types of algorithms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zLaKarYx3unc21nQO1XPRw.jpeg" /><figcaption>Types of reinforcement learning</figcaption></figure><p>It’s important to know about these types because each type has its own way of learning.</p><h4>Value-based</h4><p>Estimate the optimal value function <em>Q* (s, a).</em></p><p>This is the maximum value achievable under any policy.</p><h4>Policy-based</h4><p>Search directly for the optimal policy <em>π*</em>.</p><p>This is the policy achieving the maximum future reward.</p><h4>Actor-Critic</h4><p>We combine Value-based and Policy-based methods.</p><p>The Actor is policy-based and Critic is value-based.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*Gnx9eyqoZdyi7G5I" /><figcaption><a href="https://medium.com/free-code-camp/an-intro-to-advantage-actor-critic-methods-lets-play-sonic-the-hedgehog-86d6240171d">An intro to Advantage Actor Critic methods: let’s play Sonic the Hedgehog!</a></figcaption></figure><p>For this project I used an Actor-Critic Method. The algorithm choosed is DDPG (Deep Deterministic Policy Gradient).</p><p>DDPG works a little bit different comparing with the basic approach of Reinforcement Learning. We’ll have <strong>two</strong> neural networks inside the agent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/0*Mmhgtkak888G6bW7" /></figure><p>Now, let’s know more about the environment and the double-jointed arm!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*Kiqh2B0BiH382Pmf.gif" /></figure><p>Well, the environment that we have here it’s better than we have in the GIF above. I used Reacher Environment from Unity.</p><h3>Environment</h3><p>In this environment, a double-jointed arm can move to target locations.</p><p>The first version contains a single agent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*A5z6DtO3DCcnW10h" /></figure><p>The second version contains 20 identical agents, each with its own copy of the environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*egCoqwOVPGwdHKN6" /></figure><p>DDPG is a good approach for this environment because we can have more than one agent. We can learn from all agents and share the knowledge with all agents as well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*1MbQlf3VyTiOH4UF.png" /><figcaption><a href="https://www.groundai.com/project/deep-reinforcement-learning-for-multi-agent-systems-a-review-of-challenges-solutions-and-applications/1">Deep Reinforcement Learning for Multi-Agent Systems</a></figcaption></figure><h4>State Space</h4><p>The observation space consists of 33 variables corresponding to position, rotation, velocity, and angular velocities of the arm.</p><h4>Actions</h4><p>Each action is a vector with four numbers, corresponding to torque applicable to two joints. Every entry in the action vector should be a number between -1 and 1.</p><h4>Reward</h4><p>A reward of +0.1 is provided for each step that the agent’s hand is in the goal location. Thus, the goal of your agent is to maintain its position at the target location for as many time steps as possible.</p><p>To solve this environment we need to have 30+ score at last 100 episodes.</p><h3>Agent Architecture</h3><p>The agent created on this project consists of an <strong>agent</strong> with an <strong>Actor model</strong>, a <strong>Critic model</strong>, a <strong>Exploration Policy </strong>and a <strong>Memory Unit</strong>.</p><h4><strong>Agent</strong></h4><p>The agent has the methods that interacts with the environment: step(), act(), learn() and some others.</p><p>The Actor model, Critic model, Exploration Policy and Memory Unit will be part of the agent (as attributes).</p><h4><strong>Actor model</strong></h4><p>We have 3 layers on this neural network.</p><p><strong>Input layer</strong> will receive all the 33 features in the state space.</p><p><strong>Output layer</strong> will provide an array with 4 elements as requested to action space.</p><pre>Actor(<br>(fc1): Linear(in_features=33, out_features=256, bias=True)     (fc2): Linear(in_features=256, out_features=128, bias=True)     (fc3): Linear(in_features=128, out_features=4, bias=True)<br>)</pre><h4>Critic model</h4><p>For the Critic Neural Network we also have 3 layers.</p><p><strong>Input layer</strong> will receive all the 33 features in the state space.</p><p><strong>Output layer</strong> will provide just one value to guide the learn of Actor Model.</p><pre>Critic(<br>(fc1): Linear(in_features=33, out_features=256, bias=True)<br>(fc2): Linear(in_features=260, out_features=128, bias=True)<br>(fc3): Linear(in_features=128, out_features=1, bias=True)<br>)</pre><p>These neural networks was developed with <a href="https://pytorch.org/">https://pytorch.org/</a></p><h4><strong>Memory Unit</strong></h4><p>With two simple methods add() and sample() this memory can storage the experiences and also can returns aleatory some experiences to be used in agent training.</p><h4>Exploration Policy</h4><p>The Exploration Policy it’s important to our agent because it’s will help the agent try several actions results and avoid to stop on a local minima.</p><p>My Exploration Policy is based on <a href="https://en.wikipedia.org/wiki/Ornstein%E2%80%93Uhlenbeck_process">Ornstein-Uhlenbeck process</a>.</p><p>Now, let’s see how our DDPG algorithm with multi agents solve this environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*VYgHUJQ33J7xpc4T.gif" /><figcaption>My arm training to be the Sly!</figcaption></figure><p>Multi agent approach take a long time in each episode, you know, we have several calculations to do and two neural networks to learn, but we finished this environment with 56 episodes!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/0*VEZK1N9ddFNdjRSP" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*FeRpEjQChGa2hzii.gif" /><figcaption>My agent when we finished the environment</figcaption></figure><p>You can find the code on my GitHub’s Repository.</p><p><a href="https://github.com/DougTrajano/ds_drl_continuous_control">DougTrajano/ds_drl_continuous_control</a></p><h3>Ideas to future work</h3><h4>New combinations of hyperparameters</h4><p>This algorithm take too much time to learn, it’s very complicated to test several combinations of hyperparameters, but we can continue with the training process to find the optimal hyperparameters.</p><h4>Implement Prioritized Experience Replay</h4><p>If you read my last article, you know that I used Prioritized Experience Replay there. This memory it’s very good to provide to the agent the experiences that we’ll have more benefits on training process, but it will increase the agent’s training time.</p><p>This prioritized memory is describes <a href="https://arxiv.org/pdf/1511.05952.pdf">here</a>.</p><h4>Change the Exploration Policy</h4><p>I used Ornstein-Uhlenbeck process as recommended by initial authors of DDPG, but the most recent papers say that we have no benefits to use Ornstein-Uhlenbeck process instead of Gaussian Noise. Probably it’s can improve the agent performance.</p><p>That’s all folks! Thank you a lot to read it and don’t forget to send your comments below!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e25529a5915b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/training-a-double-jointed-arm-with-ddpg-deep-reinforcement-learning-e25529a5915b">Training a double-jointed arm with DDPG — Deep Reinforcement Learning</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Collecting bananas with Deep Reinforcement Learning]]></title>
            <link>https://medium.com/analytics-vidhya/collecting-bananas-with-drl-agent-bdd8bf216d11?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/bdd8bf216d11</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Thu, 28 May 2020 14:00:29 GMT</pubDate>
            <atom:updated>2020-06-04T19:42:05.669Z</atom:updated>
            <content:encoded><![CDATA[<h3>Collecting bananas with Deep Reinforcement Learning agent</h3><p>Today, I’m gonna share with you an amazing algorithm that learns from zero (no needs labeled data) to collecting yellow bananas while avoiding blue bananas. It’s very nice, isn’t?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/360/1*VPvtSAbr__W8cz8QsWvM3g.gif" /></figure><p>Before we talk about the algorithm and the agent, let’s understand how reinforcement learning works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*pRZ34GzZj-w_MhVl.jpg" /><figcaption>You waiting for my quick overview about reinforcement learning</figcaption></figure><h3>How Reinforcement Learning works?</h3><p>Reinforcement Learning it’s a subclass of Machine Learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/0*HMkUrzi2aKCL_sON.jpg" /><figcaption>Types of Machine Learning</figcaption></figure><p>We will have an <strong>environment </strong>and an <strong>agent </strong>… cool</p><p>The agent provides <strong>actions </strong>to environment.</p><p>The environment return the <strong>state </strong>and <strong>reward </strong>for the agent.</p><p>The objective of the agent is <strong>maximize the cumulative reward</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XH3J9NsVlE8Jph8vR98I5w.png" /></figure><p>We have some reinforcement algorithms that work by mapping the states that lead to the best cumulative reward.</p><p>It’s can works with some simple problems (with small number of data in the state and small number of actions), but the problem starts when we have a higher number of actions or infinite number of possible actions („• ֊ •„)</p><p>Then… this gave rise to the deep reinforcement learning algorithms!!</p><p>Basically, we add a Deep Learning algorithm on that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_fT1LDztAuvVI_HF" /><figcaption>Reinforcement Learning + Deep Learning = Deep Reinforcement Learning</figcaption></figure><p>For this environment, I used a Deep Q-Learning, see below the architecture of this algorithm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/0*UaIWBl6z0iWyBACd.png" /><figcaption>Deep Q-Learning algorithm</figcaption></figure><p>Now that you know the basics of reinforcement learning and deep reinforcement learning, let’s talk about bananas!</p><h3>The challenge: Bananas Collector</h3><p>This environment is provided by Unity.</p><p>Unity is a leader in this area and their have others environment, see more about Unity <a href="https://unity3d.com/machine-learning/">here</a>.</p><p>In the video below, their explains how this environment works.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FheVMs3t9qSk%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DheVMs3t9qSk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FheVMs3t9qSk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/40043c084a871d48ea3bce772a44c2c4/href">https://medium.com/media/40043c084a871d48ea3bce772a44c2c4/href</a></iframe><h4>State space</h4><p>The state space has <strong>37</strong> dimensions and contains the agent&#39;s velocity, along with ray-based perception of objects around agent&#39;s forward direction.</p><h4>Actions</h4><p>We have <strong>4</strong> actions that the agent can take:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*Xgs5VgdjBA4ophOP8vA1Mg.png" /></figure><h4><strong>Reward</strong></h4><p>A reward of +1 is provided for collecting a yellow banana, and a reward of -1 is provided for collecting a blue banana.</p><h4>Agent architecture</h4><p>The agent created on this project consists of an <strong>agent</strong>, a <strong>Deep</strong> <strong>Q-Learning</strong> model and a M<strong>emory Unit</strong>.</p><p><strong>Agent</strong></p><p>The agent has the methods that interacts with the environment: step(), act(), learn() and some others.</p><p>The Deep Q-Learning and Memory Unit will be part of the agent (as attributes).</p><p><strong>Deep Q-Learning</strong></p><p>The function of this model is understand the states and provides better actions while the agent learns more about the environment.</p><p>The architecture of the model is too simple, it’s has an <strong>input layer</strong>, a <strong>hidden layer</strong> and then, an <strong>output layer</strong>.</p><p>This neural networks was developed with <a href="https://pytorch.org/">https://pytorch.org/</a></p><p><strong>Memory Unit</strong></p><p>For the Memory Unit we have two options that can be used:</p><ul><li>ReplayMemory</li></ul><blockquote><em>With two simple methods </em><em>add() and </em><em>sample() this memory can storage the experiences and also can returns aleatory some experiences to be used in agent training.</em></blockquote><ul><li>PrioritizedMemory</li></ul><blockquote><em>This memory is a little bit more complex, because the </em><em>sample() method doesn&#39;t returns the experiences aleatory. It’s will try to understand how experiences can be used for a better training of the agent applying weights for each experience.</em></blockquote><h3>Training and Testing the agent</h3><p>To have this environment solved, the agent must get an average score of +13.</p><p>The agent has a lot of hyperparameters, some os these are used in Deep Q-Learning model, others in Memory Unit, etc.</p><pre>agent = Agent(state_size=<strong>37</strong>, action_size=<strong>4</strong>, seed=<strong>199</strong>, nb_hidden=<strong>(64, 64)</strong>, learning_rate=<strong>0.001</strong>, memory_size=<strong>int(1e6)</strong>, prioritized_memory=<strong>False</strong>, batch_size=<strong>64</strong>, gamma=<strong>0.9</strong>, tau=<strong>0.03</strong>, small_eps=<strong>0.03</strong>, update_every=<strong>4</strong>)</pre><p>We have two trained agents (with ReplayMemory and PrioritizedMemory).</p><p>See below the training session results for each agent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*pYFmhgStlfIay2fgh68TFA.png" /><figcaption>Replay Memory</figcaption></figure><p>With <strong>Replay Memory</strong>, the agent solved this environment with 389 episodes and got about 4 min to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/475/1*xmYUpjU1FH2fj9wCOZK2QQ.png" /><figcaption>Prioritized Memory</figcaption></figure><p>With <strong>Prioritized Memory</strong>, the agent solved this environment with 370 episodes and got about 11 hours and 30 minutes to learn.</p><p>We’ll test the trained agent with Prioritized Memory in the environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*-ndHCNfRFDWMadbqb3WHRg.gif" /><figcaption>Testing the agent with Prioritized Memory</figcaption></figure><p>You can find the code on my GitHub’s Repository.</p><p><a href="https://github.com/DougTrajano/ds_drl_bananas_collector">DougTrajano/ds_drl_bananas_collector</a></p><h3>Considerations</h3><p>I had a lot of fun working with this algorithm, it’s some complex at first, but I think that you’ll enjoy it as well.</p><p>If you study something about neuroscience, you’ll see some relationships about it. Our memory works similar to Prioritized Memory when we “select” some experiences to use to evaluate the options and decisions that we make every day.</p><p>For the future work I think that we can test a dueling DQN to improve this model. See more about it <a href="https://arxiv.org/abs/1511.06581">here</a>.</p><p>That’s all folks! Thank you for reading this and I appreciate if you can share your comment with me below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/1*le6e-L3vVsubEnNkIJ3KJQ.gif" /></figure><h3>References</h3><ul><li><a href="https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893">Deep Reinforcement Learning Nanodegree - Udacity</a></li><li><a href="https://arxiv.org/pdf/1511.05952.pdf">Prioritized Experience Replay - Google DeepMind</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bdd8bf216d11" width="1" height="1" alt=""><hr><p><a href="https://medium.com/analytics-vidhya/collecting-bananas-with-drl-agent-bdd8bf216d11">Collecting bananas with Deep Reinforcement Learning</a> was originally published in <a href="https://medium.com/analytics-vidhya">Analytics Vidhya</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[IA applied in Credit Risk — Home Credit]]></title>
            <link>https://dougtrajano.medium.com/ia-applied-in-credit-risk-home-credit-b70412ef8f02?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/b70412ef8f02</guid>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[credit]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Sun, 06 Oct 2019 17:02:52 GMT</pubDate>
            <atom:updated>2019-10-18T11:51:40.967Z</atom:updated>
            <content:encoded><![CDATA[<h3>IA applied in Credit Risk — Home Credit</h3><p>This article is part of my last project on <a href="https://www.udacity.com/course/data-scientist-nanodegree--nd025">Data Scientist nanodegree</a> at <a href="https://www.udacity.com/">Udacity</a>!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*u1i0Zo9iPJ-72EwG" /><figcaption>Udacity — Data Scientist nanodegree</figcaption></figure><p>The Capstone Project challenge us to test all our data science skills that we learned on the program solving real-world problems.</p><p>I thought a few days about which project I would like to do. Then, I took the best amazing ultra advanced technology to organize the work. A paper! \o/</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UTwQlCwrnfTW_5t58zd9DQ.jpeg" /><figcaption>Somethings in a paper</figcaption></figure><p>I choosed a <a href="https://www.kaggle.com/c/home-credit-default-risk/data">dataset from Kaggle</a> that was provided by Home Credit for a competition.</p><h3>Home Credit</h3><p>Many people struggle to get loans due to insufficient or non-existent credit histories. And, unfortunately, this population is often taken advantage of by untrustworthy lenders.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/668/0*SYQSbVpc2Mk82Ko1.jpg" /></figure><p><a href="http://www.homecredit.net/">Home Credit</a> strives to broaden financial inclusion for the unbanked population by providing a positive and safe borrowing experience. In order to make sure this underserved population has a positive loan experience, Home Credit makes use of a variety of alternative data — including telco and transactional information — to predict their clients’ repayment abilities.</p><p>While Home Credit is currently using various statistical and machine learning methods to make these predictions, they’re challenging Kagglers to help them unlock the full potential of their data. Doing so will ensure that clients capable of repayment are not rejected and that loans are given with a principal, maturity, and repayment calendar that will empower their clients to be successful.</p><blockquote>This description was taken from <a href="https://www.kaggle.com/c/home-credit-default-risk">here</a>.</blockquote><h3>The strategy</h3><p>Home Credit provided some csv files in the competition. I’ll use only the dataset that resume all infos about customers, but to get better results, we need to check more information on others datasets.</p><p>You can see the description of each file in my <a href="https://www.kaggle.com/dougtrajano/home-credit-default-risk/">Kaggle Notebook</a>.</p><p>The dataset have 122 characteristics from 307511 loans. Let’s see some graphs about the principal features of this dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*XZqvKX5ikCvkGGHBGFBfPA.png" /><figcaption>Loan Repayed or not</figcaption></figure><p>The ability to repayed or not the loan it’s what we are trying to predict. See that we have 92% of the dataset with <strong>TARGET:0</strong> and only 8% with <strong>TARGET:1.</strong></p><p>It’s a unbalanced class problem and we need to handle with it when build the models.</p><p>To solve this problem, I broke the project into 5 steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V3FQj7ZbB5-MlOEJ1GmimQ.png" /></figure><p>As you can see, we have a classification problem here. I’ll start creating some plots to understand the data. Then, I’ll apply some techniques to transform the data, check for missing values, etc.</p><p>Probably we need to reduce the dataset, we can do that understand how features are important and how not.</p><p>With the dataset prepared, we’ll create some models. I’ll start with simple algorithms, like LogisticRegression, Random Forest, and try more complex algorithms if needed.</p><p>If possible, we’ll apply GridSearchCV to find best parameters and apply cross-validation technique.</p><p>To evaluate our models we’ll use F1-Score and Kaggle’s Score. These scores can works better with our unbalanced classes problem.</p><blockquote><em>Kaggle’s Score are provided when we submit our predicts for the competition.</em></blockquote><h3>Data understanding</h3><p>Now that we have a clear estrategy what I think that can works with this problem, let’s see some infos about the dataset.</p><p>I really like to use <a href="https://pandas-profiling.github.io/pandas-profiling/docs/">pandas-profiling</a> to quick analysis datasets. This dataset it’s too big that Kaggle Notebook can take 83 hours to run it.</p><p>I ran the ProfileReport on AWS <a href="https://aws.amazon.com/sagemaker/">SageMaker</a> with an instance type: ml.m5.24xlarge, vCPU: 96, Mem (GiB): 384GB.</p><p>You can see the complete Dataset overview <a href="https://dougtrajano.github.io/udacity_capstone_project/dataset_overview.html">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cdk7JxmP6FXmaOgA6OiWJw.png" /><figcaption><a href="https://dougtrajano.github.io/udacity_capstone_project/dataset_overview.html">Dataset Overview</a></figcaption></figure><p>Let’s continue seeing some features distribution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*rJBDATPsHT5fVSKMLsYnjQ.png" /><figcaption>Contract type</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*fF7gR2nM8kHu5hp5sj7dmA.png" /><figcaption>Family status</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*qleZK-bw1e8ZcnojIcthXQ.png" /><figcaption>Occupation of applicant’s</figcaption></figure><h4>PCA</h4><p><a href="https://en.wikipedia.org/wiki/Principal_component_analysis">Principal Component Analysis</a> (PCA) is a technique used to emphasize variation and bring out strong patterns in a dataset.</p><p>We can use it as dimensionality reduction technique.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/621/1*mwGPB7q8cwBYa4lx_G_J_A.png" /><figcaption>PCA</figcaption></figure><h4>Relevant features</h4><p>Using the pearson correlation we can check what features are more important to understand the TARGET.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/359/1*5e-_8HYaXet6DiK4HzmE-A.png" /><figcaption>Relevant features</figcaption></figure><p>Now that we know what are the relevant features for TARGET, let’s see some graphs to better visualization.</p><p>You will see several graphs that show how the TARGET distributes itself over the relevant features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rEVxwwxl5oPR8s45XqSD3w.png" /><figcaption>Seaborn’s pairplot for relevant features</figcaption></figure><h3>Modeling</h3><p>I tried some algorithms that can works with unbalanced classes, because we have 92% of the dataset for <strong>TARGET:0 </strong>and only 8% of the dataset for <strong>TARGET:1</strong>.</p><h4>Logistic Regression</h4><p>Logistic Regression is a simple machine learning algorithm. It’s very similar to Linear Regression, let’s see the difference between Linear Regression and Logistic Regression below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/0*-srsnacJAQe8DJGq.png" /><figcaption>Linear Regression and Logistic Regression from <a href="https://medium.com/@ODSC/logistic-regression-with-python-ede39f8573c7">ODSC’s post</a>.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/1*Gh3FwguUupnke-07k3lUkw.png" /><figcaption>ClassificationReport — LogisticRegression (basic parameters)</figcaption></figure><p>We can see that our results are not good enough using LogisticRegression.</p><p>We got 68.98% of accuracy and 52.68% of F1-Score (macro). :(</p><h4>Random Forest</h4><p>I really like this algorithm, it’s my favorite algorithm and I always try to solve problem of classification with it.</p><p>This algorithm create a model based on DecisionTrees and each DecisionTree try to understand a small piece of the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/0*yQxIv9AgWA8oiZzU.png" /><figcaption>Random Forest Simplified from <a href="https://medium.com/@williamkoehrsen/random-forest-simple-explanation-377895a60d2d">williamkoehrsen’s post</a>.</figcaption></figure><p>Random Forest works little better than LogisticRegression with basic parameters</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/476/1*ECF1ViRv3TWlGE0MR-kumg.png" /><figcaption>Classification Report — Random Forest (basic parameters)</figcaption></figure><h4>LGBMClassifier</h4><p>It’s a new machine learning algorithm. This algorithm will create a gradient boosting model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fg5u37sAcUyF73Fh.png" /><figcaption>LightGBM from <a href="https://medium.com/@pushkarmandot/https-medium-com-pushkarmandot-what-is-lightgbm-how-to-implement-it-how-to-fine-tune-the-parameters-60347819b7fc">pushkarmandot’s post</a></figcaption></figure><p>LGBMClassifier obtained similar results to Random Forest with basic parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*fl9WG7HZ3YnoDIY9wBookw.png" /><figcaption>Classification Report — LGBMClassifier (basic parameters)</figcaption></figure><h4><strong>Keras</strong></h4><p>Keras it’s a great lib to Deep Learning models, I utilize Sequential model from Keras to create a Multilayer Perceptron (MLP) model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/567/0*8YNmHx4zDylzdeMi.png" /><figcaption>Keras Sequential model from <a href="https://medium.com/@subhamoy.paul986/keras-sequential-api-72e45c39259b">subhamoy.paul986’s post</a>.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/267/1*vld7hjPVlxIrvGAF-ox0_Q.png" /><figcaption>Classification Report — Keras Model</figcaption></figure><p>Keras didn’t perform properly in this dataset, but I think that the problem is my basic skills with this algorithm.</p><p>We got 72.54% of accuracy and 23.80% of F1-Score (macro).</p><h3>Refinement</h3><p>We got better results using Random Forest and LGBMClassifier than Logistic Regression and Keras, then I decided apply GridSearchCV to find best parameters for each of these models.</p><h4>Random Forest</h4><pre>parameters = {<br>    &quot;n_estimators&quot;: [10, 30, 50, 100],<br>    &quot;criterion&quot;: [&quot;gini&quot;, &quot;entropy&quot;],<br>    &quot;max_depth&quot;: [10, 20, 30],<br>    &quot;min_samples_leaf&quot;: [50, 100, 200]    <br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/1*fN65JCYavNYHkAs-Eu9vCg.png" /><figcaption>Classification Report — Random Forest (GridSearchCV)</figcaption></figure><p>We got 78.92% (improvement 6.94%) of accuracy and 57.65% (improvement 3.46%) of F1-Score (macro) with the best estimator.</p><h4>LGBMClassifier</h4><pre>parameters = {<br>    &quot;n_estimators&quot;: [100, 300, 500, 1000],<br>    &quot;boosting_type&quot;: [&quot;gbdt&quot;, &quot;dart&quot;, &quot;goss&quot;],<br>    &quot;max_depth&quot;: [10, 20, 30] <br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*5D4CcTqCLq1m-vSgEzfEfQ.png" /><figcaption>Classification Report — LGBMClassifier (GridSearchCV)</figcaption></figure><p>We got 81.14% (improvement 9.66%) of accuracy and 58.28% (improvement 4.05%) of F1-Score (macro).</p><h3>Results</h3><p>After get the properly hyperparameters for each model, I sent the predictions to Kaggle. Let’s see below the Kaggle Score for each algorithm that I tried here.</p><blockquote>The Kaggle Score for this competition use ROC AUC score.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*wtQiADcUh-Q3QgqJOy5gJA.png" /><figcaption>Kaggle Score for each algorithm</figcaption></figure><p>We can see that Random Forest performs better than LGBMClassifier in the real world. It’s my first experience with LGBMClassifier, but I always works with Random Forest to solve this type of problem. I think that we can try more parameters and a better data preparation to get more results with this algorithm.</p><h3>Improvements</h3><p>To improve this study I think that we can use others csv files provided by HomeCredit, it’s can result in better scores because we have csv files with bureau data. Bureau data has application from previous loans that client got from other institutions and that were reported to Credit Bureau.</p><p>Another thing is test other algorithms and try to got more data for target 1, perhaps applying data augmentation technique.</p><h3>Conclusion</h3><p>To summarize, we started with understanding the data, applying techniques to handle with missing values, PCA, relevant features, etc. Then, we create some models and to finalize, we send files with predictions to get the real score from Kaggle Competition.</p><p>This is a complex problem and we need more experimentation to find a better solution, perhaps, more features to achieve better scores.</p><p>I’m happy to create this study because I learned more about credit risk and data science skills like TensorFlow and Keras. I worked with a new algorithm that I never worked before (LGBMClassifier).</p><p><a href="https://github.com/DougTrajano/udacity_capstone_project">GitHub Repo</a></p><p><a href="https://www.kaggle.com/dougtrajano/home-credit-default-risk">Kaggle Notebook</a></p><p>Thank you for read it and feel free to comment your opinion and suggestion about this work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*upds9oU3Sf4AGMZCCJ6s0g.gif" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b70412ef8f02" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Analysis about Stack Overflow survey]]></title>
            <link>https://dougtrajano.medium.com/analysis-about-stack-overflow-survey-244dac0b00fe?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/244dac0b00fe</guid>
            <category><![CDATA[stackoverflow]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Thu, 18 Jul 2019 03:14:35 GMT</pubDate>
            <atom:updated>2019-07-18T03:14:35.647Z</atom:updated>
            <content:encoded><![CDATA[<p>Hi! This is my first post and it’s part of my Data Scientist nanodegree from Udacity! \o/</p><p>The objective is choose a dataset and explore insights from that.</p><p>I choosed a dataset from Stack Overflow’s survey because I understand that it’s has rich information. The dataset can be found on <a href="https://www.kaggle.com/stackoverflow/so-survey-2017">kaggle</a>.</p><h3>Dataset information</h3><pre>Number of variables: <strong>154<br></strong>- Numeric: <strong>7</strong><br>- Categorical: <strong>147</strong></pre><pre>Number of observations: <strong>51392</strong></pre><pre>Total Missing (%): <strong>45.3%</strong></pre><h3>Business understanding</h3><p>Stack Overflow it’s a developer forum, it’s natural that the most of users are developers, but we can see a part of users that are students.</p><figure><img alt="Distribuição de profissionais ou não na pesquisa do Stack Overflow" src="https://cdn-images-1.medium.com/max/871/1*NMUnqGq2Cu4HefFllEyoqg.png" /><figcaption>Users distribution</figcaption></figure><p>The main idea that I’ll works it’s compare professional developers with students.</p><h3>University</h3><p>Have a degree it’s important to be a developer?</p><p>82,5% of professional developers aren’t in a university currently.</p><figure><img alt="University — Professional developer" src="https://cdn-images-1.medium.com/max/700/1*Cd3cGLpP5oOzwGMWjwMyUA.png" /><figcaption>University distribution in professional developers</figcaption></figure><p>It’s make sense when we take a look on them education status, 75% of these developers has a degree.</p><figure><img alt="Educação formal para desenvolvedores profissionais" src="https://cdn-images-1.medium.com/max/960/1*NztU4-JbR5aANWaSp5ynuA.png" /><figcaption>Education status in professional developers</figcaption></figure><p>Talking about students, 63% of these students are in a university, full-time or part-time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/1*Lqsd3behUCuhQuTAXz92CQ.png" /><figcaption>Education status in students</figcaption></figure><p>We can understand that most of developers finished their degrees and the most of students are in enrolled in a degree currently.</p><h3>Work status</h3><p>We will compare the work status between professional developers and students.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/859/1*ec-7_gARy_KtSAywyvLCxQ.png" /><figcaption>Employment status in developers</figcaption></figure><p>97,74% of developers has a work relationship currently.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*zizWTJEB6j0ZfgJk39qdew.png" /><figcaption>Employment status in students</figcaption></figure><p>When we analysed the students, we have different results. 39,78% are employed. 28,1% aren’t in employed and aren’t searching for.</p><p>We have a significant different between users that prefer not to say in students (8,56%) and developers (0,53%).</p><h3>Salary and expected salary</h3><p>Now we will understand more about the developers salaries and the expected salaries of students.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*0Lb7FXGzlBZzaK6DFMLOdQ.png" /><figcaption>Salário entre desenvolvedores profissionais (histograma)</figcaption></figure><p>The most of developers has a salary between USD 25k and USD 65k, but exists some variables that can influence in the salary that a developer receive, as example, the years of experience. See below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*Sp_GIAUfH5_hMVMej8Qv5Q.png" /><figcaption>Developers salaries based on years of experience</figcaption></figure><p>The more experience a developer has, more high is the salary that he receive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/0*JprAj0oG1AeQvdsh.gif" /></figure><p>Now, take a look on students, we will analyse their expected salaries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*ScrjTEqWQAhPG1UKbrGs1A.png" /><figcaption>Expectativa de salário dos estudantes (histograma)</figcaption></figure><p>It seems that we have some misunderstanding of the question that was asked, since most of the users put an annual remuneration of USD 5k, if we ignore this, the students are expecting salaries between USD 25k and USD 55k. According to what is offered by the market.</p><h3>Conclusions</h3><p>A quickly analyse in this dataset can give us the understanding about students and developers of stack overflow.</p><p>As next step of this analyse I suggest the variations of results between more variables, as example, if we check the mean salary of developers based on countries we can see a big diference.</p><p>The developers from USA has a salary about USD 100k while Indian developers has salaries of USD 10k.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/215/1*LlJWdQkjOp_P3GVVRFFNQw.png" /><figcaption>Mean salary</figcaption></figure><p>Well, if you’re a student and want to become a developer, it’s better if you are living on USA or Australia. \o/</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*cOFnthLvjyPTaE73.gif" /></figure><p>The source code is available on my GitHube <a href="https://github.com/DougTrajano/ds-study_stack-overflow-survey">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=244dac0b00fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Análise do Stack Overflow]]></title>
            <link>https://dougtrajano.medium.com/an%C3%A1lise-do-stack-overflow-dfb0b5ade16e?source=rss-e513ae1eef97------2</link>
            <guid isPermaLink="false">https://medium.com/p/dfb0b5ade16e</guid>
            <category><![CDATA[stackoverflow]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Douglas Trajano]]></dc:creator>
            <pubDate>Mon, 15 Jul 2019 02:40:17 GMT</pubDate>
            <atom:updated>2019-07-15T02:40:17.158Z</atom:updated>
            <content:encoded><![CDATA[<p>Olá! Este é meu primeiro post e faz parte da minha formação no Data Science nanodegree da Udacity!</p><p>O objetivo é escolher um dataset, analisá-lo e gerar insights relevantes sobre o tema escolhido.</p><p>Escolhi um dataset de uma pesquisa do Stack Overflow por entender que este dataset é rico em informações. O dataset pode ser encontrando no <a href="https://www.kaggle.com/stackoverflow/so-survey-2017">kaggle</a>.</p><h3>Informações do dataset</h3><pre>Number of variables: <strong>154<br></strong>- Numeric: <strong>7</strong><br>- Categorical: <strong>147</strong></pre><pre>Number of observations: <strong>51392</strong></pre><pre>Total Missing (%): <strong>45.3%</strong></pre><h3>Entendendo o cenário</h3><p>Stack Overflow é um fórum para desenvolvedores, é natural que o maior público do site seja desenvolvedores, mas também há uma parcela de estudantes que acessam a plataforma.</p><figure><img alt="Distribuição de profissionais ou não na pesquisa do Stack Overflow" src="https://cdn-images-1.medium.com/max/871/1*NMUnqGq2Cu4HefFllEyoqg.png" /><figcaption>Distribuição de usuários</figcaption></figure><p>A ideia que irei abordar é comparar os profissionais que já são desenvolvedores com os estudantes.</p><h3>Universidade</h3><p>Será que realizar uma graduação é importante para se tornar um desenvolvedor?</p><p>82,5% dos desenvolvedores profissionais não estão em uma universidade atualmente.</p><figure><img alt="University — Professional developer" src="https://cdn-images-1.medium.com/max/700/1*Cd3cGLpP5oOzwGMWjwMyUA.png" /><figcaption>Universidade entre desenvolvedores profissionais</figcaption></figure><p>Faz sentido quando olhamos a formação deles, 75% deles já possuem graduação e/ou pós-graduação.</p><figure><img alt="Educação formal para desenvolvedores profissionais" src="https://cdn-images-1.medium.com/max/960/1*NztU4-JbR5aANWaSp5ynuA.png" /><figcaption>Educação formal para desenvolvedores profissionais</figcaption></figure><p>Falando dos estudantes agora, 63% dos estudantes estão em uma universidade, seja em tempo integral ou parcial.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/735/1*Lqsd3behUCuhQuTAXz92CQ.png" /><figcaption>Educação formal entre estudantes</figcaption></figure><p>Podemos entender que a maioria dos desenvolvedores já se formaram e que a maioria dos estudantes ainda estão realizando sua graduação.</p><h3>Situação atual do trabalho</h3><p>Vamos comparar a situação atual de trabalho dos desenvolvedores profissionais com os estudantes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/859/1*ec-7_gARy_KtSAywyvLCxQ.png" /><figcaption>Empregabilidade entre desenvolvedores profissionais</figcaption></figure><p>97,74% dos desenvolvedores possuem alguma relação de trabalho atualmente.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/826/1*zizWTJEB6j0ZfgJk39qdew.png" /><figcaption>Empregabilidade entre estudantes</figcaption></figure><p>Quando analisamos os estudantes, temos resultados bem diferentes. 39,78% estão atualmente empregados. 28,1% não estão empregados e não estão procurando.</p><p>Temos uma diferença significativa entre os usuários que preferiram não falar sua situação atual (8,56%) se compararmos com os desenvolvedores profissionais (0,53%)</p><h3>Salário e expectativa de salário</h3><p>Agora vamos entender mais sobre o salário dos desenvolvedores profissionais e da expectativa de salário dos estudantes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*0Lb7FXGzlBZzaK6DFMLOdQ.png" /><figcaption>Salário entre desenvolvedores profissionais (histograma)</figcaption></figure><p>A maior parte dos desenvolvedores recebem entre USD 25k e USD 65k, porém existem algumas variáveis que influenciam o salário que um desenvolvedor profissional pode receber, como o tempo de experiência. Vejamos abaixo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*Sp_GIAUfH5_hMVMej8Qv5Q.png" /><figcaption>Salário entre desenvolvedores profissionais (tempo de experiência)</figcaption></figure><p>Quanto mais tempo de experiência um desenvolvedor tem, mais ele ganha.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/380/0*JprAj0oG1AeQvdsh.gif" /></figure><p>Agora, vejamos os estudantes, vamos analisar a expectativa de salário que eles possuem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*ScrjTEqWQAhPG1UKbrGs1A.png" /><figcaption>Expectativa de salário dos estudantes (histograma)</figcaption></figure><p>Parece que temos alguns erros de entendimento da pergunta que foi feita, pois a maior parte dos usuários colocaram como objetivo uma remuneração anual de USD 5k, se ignorarmos isto, os estudantes estão esperando salários entre USD 25k e USD 55k. De acordo com o que é oferecido pelo mercado.</p><h3>Considerações</h3><p>Uma rápida análise nos dados nos permitiu entender mais sobre os estudantes e os profissionais que já atuam como desenvolvedores.</p><p>Como continuação deste estudo acredito que devemos observar variações nos resultados de acordo com o país dos usuários, pois podemos ver diferença na média salarial dos profissionais de acordo com os países.</p><p>Os desenvolvedores nos estados unidos recebem aproximadamente USD 100k enquanto desenvolvedores na Índia recebem USD 10k.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/215/1*LlJWdQkjOp_P3GVVRFFNQw.png" /><figcaption>Média salarial</figcaption></figure><p>Bom, caso você seja estudante e queira se tornar um desenvolvedor, que seja nos EUA ou na Australia. \o/</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*cOFnthLvjyPTaE73.gif" /></figure><p>O material de estudo está disponível no meu <a href="https://github.com/DougTrajano/">GitHub</a> <a href="https://github.com/DougTrajano/ds-study_stack-overflow-survey">aqui</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dfb0b5ade16e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>