<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pablo Maldonado on Medium]]></title>
        <description><![CDATA[Stories by Pablo Maldonado on Medium]]></description>
        <link>https://medium.com/@pablito_prague?source=rss-17ce4d4503b1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*veWMn4ZHEjHQca_7.jpg</url>
            <title>Stories by Pablo Maldonado on Medium</title>
            <link>https://medium.com/@pablito_prague?source=rss-17ce4d4503b1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 05:34:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pablito_prague/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Building a data science team that works]]></title>
            <link>https://medium.com/@pablito_prague/building-a-data-science-team-that-works-328eba5dcf70?source=rss-17ce4d4503b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/328eba5dcf70</guid>
            <dc:creator><![CDATA[Pablo Maldonado]]></dc:creator>
            <pubDate>Sun, 09 Dec 2018 19:21:48 GMT</pubDate>
            <atom:updated>2018-12-09T19:21:51.739Z</atom:updated>
            <content:encoded><![CDATA[<p>Many companies are trying to build data science capabilities, for various reasons: either they saw that a competitor is extracting real value, or, more commonly, simple fear of missing out.</p><p>Sometimes these initiatives fail. Actually, rather often. But why is that and how to prevent it? In my experience, there are three key factors that contribute to the success.</p><h4>1. Strong internal buy-in</h4><p>Your forthcoming data science team must have a clear sponsor on the organization. This person is ideally someone high enough in the food chain to understand what the real issues and pain-points are. Why is that? There needs to be a clear definition of success. You can understand success if you don’t understand where problems are. The data science team is as good as the problems it can actually solve.</p><p>More importantly, KPIs should be defined: when is a data science initiative declared successful? After the number of lines of code produced? Number of algorithms in production? Bottom-line benefits? New businesses?</p><h4>2. The right people.</h4><p>You must plan your hiring for the right profile according to the maturity level of your organization. Do you want to run deep learning in GPUs and just bought a Hadoop cluster? Hold on, let’s talk first about your reporting and your data quality.</p><p>Popular opinion makes us believe that you need AI PhDs from top-level universities that code the deepest deep learning in and must get paid dearly. Actually, more often than not, these people are useless: they don’t know your business and they don’t care to learn it. Their interest is, at best, purely on the algorithms and, at worst, on getting that juicy salary only.</p><p>How to weed out the bad candidates? Either you hire one of those AI PhDs and make sure that he/she is super-passionate about your business. I cannot stress this enough: <strong><em>make sure they love your business!</em></strong> Otherwise you are in for leaving a trail of disappointed people: your flagrant new hire and your senior stakeholders.</p><p>An even better idea is to <strong><em>train and encourage</em></strong> your in-house experts on developing their own data science skills. These people have been with you, they know the business from inside out and are presumably passionate enough to be working on it.</p><p>On the very early stages, when your data science initiative is about grabbing the low-hanging fruit, <strong><em>well-trained domain experts can get you up and running with insights faster</em></strong>.</p><h4>3. Technology in place (and a good work environment!)</h4><p>It should be somewhat obvious, but I have been in many data science projects where the data takes some time to come. Waiting 2–3 weeks for getting clearance to access all the necessary data is the rule in many large companies. This is, of course, not ideal, but it does not even stop there: data scientists often face terrible laptops, no permissions to install software updates or packages (Python 3? Forget it) and sometimes not even internet access. Our team was once asked to develop an entity reconciliation engine using only data from the test environment (yes, really) because of confidentiality issues. These blocks are sufficient to discourage the most enthusiast and prepared data science team, and will pile up in the failed IT projects in the organization.</p><p>As the team grows in size and, more importantly, value has been proven, some hardware purchases might be in order (GPU cluster, etc). But not before.</p><p>Last but not least, data science is a highly creative job! Any job with data demands huge concentration. You should provide your team with spaces to run away and dive in the data, if you cannot avoid the open office. Putting your data science team next to the hectic sales team will not do.</p><p><em>Have you, or perhaps are you building your own data science team? What are your experiences? Let me know in the comments!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=328eba5dcf70" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reinforcement Learning in Python, Part 1: Welcome to RL]]></title>
            <link>https://medium.com/@pablito_prague/reinforcement-learning-in-python-part-1-welcome-to-rl-86ecf9436d77?source=rss-17ce4d4503b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/86ecf9436d77</guid>
            <dc:creator><![CDATA[Pablo Maldonado]]></dc:creator>
            <pubDate>Thu, 19 Jul 2018 09:44:42 GMT</pubDate>
            <atom:updated>2018-07-19T09:44:45.288Z</atom:updated>
            <content:encoded><![CDATA[<p>Reinforcement Learning is a framework for tackling <strong>sequential decision problems</strong>: what to do next in order to maximize a reward (which might be delayed), on a changing universe (which might react to our actions).</p><p>Examples of this include:</p><ul><li>Game playing: which actions are critical to win a game?</li><li>Learning in a “small world”: what actions maximize pleasure / minimize pain?</li><li>Treatment of chronical diseases: how to evolve the treatment for a disease that creates resistance?</li></ul><p>The unifying theme on the problems above can be abstracted as follows:</p><ul><li>An <strong>agent</strong> receives a signal from the environment, selected by Nature.</li><li>The agent executes an <strong>action</strong>.</li><li>Given the agents’ action, Nature assigns a reward and draws a new state, which is announced to the agent.</li><li>The situation repeats until a terminal criterion is reached.</li></ul><h4>An example: OpenAI Gym</h4><p>We’ll use the <a href="https://github.com/openai/gym">OpenAI Gym</a> environment for this.</p><p>It consists of a number of Atari environments that we can use for experimenting during this course. If you haven’t please install the library OpenAI gym (pip install gym).</p><pre>from tqdm import tqdm<br>import gym<br><br>#create a single game instance<br>env = gym.make(&quot;FrozenLake-v0&quot;)</pre><p>Here, <strong>S</strong> is the initial state, and your aim is to reach <strong>G</strong>, without falling into the holes, <strong>H</strong>. The squares marked with <strong>F</strong> are frozen, which means you can step on them.</p><p><strong>Note:</strong> The environment is non-deterministic, you can slip in the ice and end up in a different state.</p><p>How to use the environment?</p><ul><li><strong>reset()</strong> returns the initial state / first observation.</li><li><strong>render()</strong> returns the current environment state.</li><li><strong>step(a)</strong> returns what happens after action a:</li><li><em>new observation</em>: the new state.</li><li><em>reward</em>: the reward corresponding to that action in that state.</li><li><em>is done</em>: binary flag, True if the game is over.</li><li><em>info</em>: Some auxiliary stuff, which we can ignore now.</li></ul><p>The code below is self-explanatory. If you run the three blocks within print statements in a Jupyter notebook it will make a lot of sense :)</p><pre>print(&quot;The initial state: &quot;, env.reset())<br>print(&quot; and it looks like: &quot;)<br>env.render()<br><br>The initial state:  0 and it looks like: <br><br>SFFF<br>FHFH<br>FFFH<br>HFFG<br><br>print(&quot;Now let&#39;s take an action: &quot;)<br>new_state, reward, done, _ = env.step(1)<br>env.render()<br><br><br>idx_to_action = {<br>    0:&quot;&lt;&quot;, #left<br>    1:&quot;v&quot;, #down<br>    2:&quot;&gt;&quot;, #right<br>    3:&quot;^&quot; #up<br>}<br><br>Now let&#39;s take an action: <br>  (Down)<br>SFFF<br>FHFH<br>FFFH<br>HFFG</pre><p>Defining a policy</p><ul><li>The environment has a 4x4 grid of states.</li><li>For each state there are 4 possible actions.</li></ul><p>A <strong>policy</strong> is a function from states to actions. It tells us what we should do on each state. In this case, any array of size 16x4 is a (deterministic) policy.</p><p>We can implement policies as dictionaries.</p><pre>import numpy as np<br>n_states = env.observation_space.n<br>n_actions = env.action_space.n<br><br># Initialize random_policy:<br>def init_random_policy():<br>    random_policy  = {}<br>    for state in range(n_states):<br>        random_policy[state] = np.random.choice(n_actions)<br>    return random_policy</pre><p>We need now to define a function to evaluate our policy.</p><pre>def evaluate(env, policy, max_episodes=100): <br>    tot_reward = 0<br>    for ep in range(max_episodes):<br>        state = env.reset()<br>        done = False<br>        ep_reward = 0<br>        <br>        # Reward per episode<br>        while not done:<br>            action = policy[state]<br>            new_state, reward, done, _ = env.step(action)<br>            ep_reward += reward<br>            state = new_state<br>            if done:<br>                tot_reward += ep_reward<br>    return tot_reward/(max_episodes)</pre><p>Looking for the best policy: Random search</p><p>As a very first example, let’s try to find our policy by pure random search: we will play for some time and keep track of the best actions we can do on each state.</p><p>FrozenLake-v0 defines “solving” as getting average reward of 0.78 over 100 consecutive trials.</p><pre>best_policy = None<br>best_score = -float(&#39;inf&#39;)<br><br># Random search<br>for i in range(1,10000): #tip: you can use tqdm(range(1,10000)) for a progress bar<br>    policy = init_random_policy()<br>    score = evaluate(env,policy,100)<br>    if score &gt; best_score:<br>        best_score = score<br>        best_policy = policy<br>    if i%5000 == 0:<br>        print(&quot;Best score:&quot;, best_score)<br>print(&quot;Best policy:&quot;)<br>print(best_policy)</pre><p>running the code above we get:</p><pre>Best score: 0.45<br>Best policy:<br>{0: 2, 1: 3, 2: 0, 3: 1, 4: 0, 5: 3, 6: 0, 7: 3, 8: 3, 9: 1, 10: 0, 11: 1, 12: 3, 13: 2, 14: 2, 15: 1}<br># Let&#39;s see the policy in action<br>def play(env,policy, render=False):<br>    s = env.reset()<br>    d = False<br>    while not d:<br>        a = policy[s]<br>        print(&quot;*&quot;*10)<br>        print(&quot;State: &quot;,s)<br>        print(&quot;Action: &quot;,idx_to_action[a])<br>        s, r, d, _ = env.step(a)<br>        if render:<br>            env.render()<br>        if d:<br>            print(r)</pre><p>Let’s create a small function to print a nicer policy:</p><pre>def print_policy(policy):<br>    lake = &quot;SFFFFHFHFFFHHFFG&quot;<br>    arrows = [idx_to_action[policy[i]] <br>               if lake[i] in &#39;SF&#39; else &#39;*&#39; for i in range(n_states)]<br>    for i in range(0,16,4):<br>        print(&#39;&#39;.join(arrows[i:i+4]))</pre><p>We can call then use the functions above to render the optimal policy. Note that the policy might not give the optimal action: recall that there is some noise involved (you can slip on the ice) which is responsible of a non-optimal action looking like optimal.</p><pre>print_policy(best_policy)<br>play(env,best_policy)</pre><h4>Using a different policy</h4><p>Let’s try some different implementation of a random policy, which will be more useful later on.</p><pre># theta = 0.25*np.ones((n_states,n_actions))<br>def random_parameter_policy(theta):<br>    theta = theta/np.sum(theta, axis=1, keepdims=True) # ensure that the array is normalized<br>    policy  = {}<br>    probs = {}<br>    for state in range(n_states):<br>        probs[state] = np.array(theta[state,:])<br>        policy[state] = np.random.choice(n_actions, p = probs[state])<br>    return policy</pre><pre>best_policy = None<br>best_score = -float(&#39;inf&#39;)<br>alpha = 1e-2<br># Random search<br>for i in range(1,10000): <br>    theta = 0.25*np.ones((n_states,n_actions))<br>    policy = random_parameter_policy(theta)<br>    score = evaluate(env,policy,100)<br>    if score &gt; best_score:<br>        best_score = score<br>        best_policy = policy<br>    theta = theta + alpha*(score-best_score)*np.ones((n_states,n_actions))<br>    if i%5000 == 0:<br>        print(&quot;Best score:&quot;, best_score)<br>print(&quot;Best policy:&quot;)<br>print(best_policy)<br>print(&quot;Best score:&quot;, best_score)</pre><p>the outcome is:</p><pre>Best score: 0.39<br>Best policy:<br>{0: 0, 1: 1, 2: 0, 3: 1, 4: 0, 5: 1, 6: 0, 7: 2, 8: 3, 9: 1, 10: 0, 11: 1, 12: 1, 13: 2, 14: 3, 15: 1}<br>Best score: 0.53</pre><p>What’s the advantage of this? Perhaps not much right now, but this is the first step to use more sophisticated techniques over random search. Note that we do a “gradient update” of sorts when we change the parameter theta in the direction of increase of the best score. This hints that we could use other update rules, perhaps taking the output as a more sophisticated input of the game history.</p><p>Another thing to notice is that we made effectively our policy <strong>stochastic</strong>: at every stage the agent has the possibility of choosing randomly his action. This has the effect of smoothing out the problem: we are now solving an optimization problem on a continuous, instead of a discrete space.</p><h4>Your turn:</h4><ul><li>Beat the hill climbing / random search benchmark! Implement a different search method for the parameters.</li><li>Try the CartPole environment. In CartPole, the state is continuous (4 different parameters), so you need to do something on the lines of the parameterized random search example. Look at this <a href="http://kvfrans.com/simple-algoritms-for-solving-cartpole/">blog post</a> for inspiration.</li></ul><p><em>Do you want to learn reinforcement learning on a live tutorial? </em><a href="https://www.safaribooksonline.com/live-training/courses/reinforcement-learning-with-tensorflow-and-keras/0636920203117/"><em>Join</em></a><em> our August 23–24 session!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86ecf9436d77" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Scrape a job listing from StackOverflow]]></title>
            <link>https://medium.com/@pablito_prague/how-to-scrape-a-job-listing-from-stackoverflow-4c7323061ae5?source=rss-17ce4d4503b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/4c7323061ae5</guid>
            <dc:creator><![CDATA[Pablo Maldonado]]></dc:creator>
            <pubDate>Thu, 05 Jul 2018 11:49:50 GMT</pubDate>
            <atom:updated>2018-07-05T11:49:53.810Z</atom:updated>
            <content:encoded><![CDATA[<p><em>In this guest post, Michael Heydt, the author of the </em><a href="https://www.packtpub.com/big-data-and-business-intelligence/python-web-scraping-cookbook"><em>Python Web Scraping Cookbook</em></a><em>, shows how to scrape a job listing from StackOverflow using Python.</em></p><h4><strong>It’s Easy to Scrape Stack Overflow</strong></h4><p>StackOverflow actually makes it quite easy to scrape data from their pages. In this article, you’ll be shown how to use content from a posting at<a href="https://stackoverflow.com/jobs/122517/spacex-enterprise-software-engineer-full-stack-spacex?so=p&amp;sec=True&amp;pg=1&amp;offset=22&amp;cl=Amazon%3b+"> https://stackoverflow.com/jobs/122517/spacex-enterprise-software-engineer-full-stack-spacex?so=p&amp;sec=True&amp;pg=1&amp;offset=22&amp;cl=Amazon%3b+</a>. You’ll note that StackOverflow job listing pages are very structured. It’s probably because they’re created by programmers and for programmers. The page looks like the following:</p><p>All of this information is codified within the HTML of the page. You can see for yourself by analyzing the page content. But what’s great about StackOverflow is that it puts much of its page data in an embedded JSON object. This is placed within a &lt;script type=”application/ld+json”&gt; HTML tag, so it’s really easy to find. The following shows a truncated section of this tag (the description is truncated, but all the tags are shown):</p><p>This makes it very easy to get hold of the content, as you can simply retrieve the page, find this tag, and then convert this JSON into a Python object with the json library. In addition to the actual job description, much of the <strong>“metadata”</strong> of the job posting is also included, such as skills, industries, benefits, and location information. You don’t need to search the HTML for the information — just find this tag and load the JSON. Please note that if you want to find specific items (such as <strong>Job Responsibilities),</strong> you’ll still need to parse the description. The description contains the full HTML, so you’ll still need to deal with HTML tags while parsing it.</p><h4><strong>Start scraping a job listing from StackOverflow</strong></h4><p>It’s time to get the job description from this page; you simply have to retrieve the contents. Here’s a step by step walkthrough of the code:</p><ol><li>First, you need to read the file:</li></ol><pre>with open(&quot;spacex-job-listing.txt&quot;, &quot;r&quot;) as file:<br>    content = file.read()</pre><ol><li>Then, load the content into a BeautifulSoup object, and retrieve the &lt;script type=”application/ld+json”&gt; tag:</li></ol><pre>bs = BeautifulSoup(content, &quot;lxml&quot;)<br>script_tag = bs.find(&quot;script&quot;, {&quot;type&quot;: &quot;application/ld+json&quot;})</pre><ol><li>Now that you have the tag, you can simply load its contents into a Python dictionary using the json library:</li></ol><pre>job_listing_contents = json.loads(script_tag.contents[0])<br>print(job_listing_contents)</pre><p>The output of this looks like the following (this is truncated for brevity):</p><pre>{&#39;@context&#39;: &#39;http://schema.org&#39;, &#39;@type&#39;: &#39;JobPosting&#39;, &#39;title&#39;: &#39;SpaceX Enterprise Software Engineer, Full <br>Stack&#39;, &#39;skills&#39;: [&#39;c#&#39;, &#39;sql&#39;, &#39;javascript&#39;, &#39;asp.net&#39;, &#39;angularjs&#39;], &#39;description&#39;: &#39;&lt;h2&gt;About this <br>job&lt;/h2&gt;\r\n&lt;p&gt;&lt;span&gt;Location options: &lt;strong&gt;Paid relocation&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;<br>&lt;span&gt;Job type: &lt;strong&gt;Permanent&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;span&gt;Experience level: &lt;strong&gt;Mid-<br>Level, Senior&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;span&gt;Role: &lt;strong&gt;Full Stack Developer&lt;/strong&gt;&lt;/span&gt;<br>&lt;br/&gt;&lt;span&gt;Industry: &lt;strong&gt;Aerospace, Information Technology, Web Development&lt;/strong&gt;</pre><ol><li>Now do some simple tasks with this without involving HTML parsing. For example, you can retrieve the skills required for the job with just a few lines of code:</li></ol><pre># print the skills<br>for skill in job_listing_contents[&quot;skills&quot;]:<br>    print(skill)</pre><p>The above produces the following output:</p><pre><strong>c#</strong><br><strong>sql</strong><br><strong>javascript</strong><br><strong>asp.net</strong><br><strong>angularjs</strong></pre><p>Reading and cleaning the description in the job listing</p><p>If you take a close look, you’ll see that the description of the job listing is still in HTML. In order to extract valuable content out of this data, you’ll need to parse this HTML and perform several other steps:</p><ol><li>Start by creating a BeautifulSoup object from the description key of the description you loaded. Print this to see what it looks like:</li></ol><pre>desc_bs = BeautifulSoup(job_listing_contents[&quot;description&quot;], &quot;lxml&quot;)<br>print(desc_bs)</pre><pre>&lt;p&gt;&lt;span&gt;Location options: &lt;strong&gt;Paid relocation&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;span&gt;Job type: &lt;strong&gt;Permanent&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;span&gt;Experience level: &lt;strong&gt;Mid-Level, Senior&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;span&gt;Role: &lt;strong&gt;Full Stack Developer&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;span&gt;Industry: &lt;strong&gt;Aerospace, Information Technology, Web Development&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;span&gt;Company size: &lt;strong&gt;1k-5k people&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;span&gt;Company type: &lt;strong&gt;Private&lt;/strong&gt;&lt;/span&gt;&lt;br/&gt;&lt;/p&gt;&lt;br/&gt;&lt;br/&gt;&lt;h2&gt;Technologies&lt;/h2&gt; &lt;p&gt;c#, sql, javascript, asp.net, angularjs&lt;/p&gt; &lt;br/&gt;&lt;br/&gt;&lt;h2&gt;Job description&lt;/h2&gt; &lt;p&gt;&lt;strong&gt;Full Stack Enterprise Software Engineer&lt;/strong&gt;&lt;/p&gt;<br>&lt;p&gt;The EIS (Enterprise Information Systems) team writes the software that builds rockets and powers SpaceX. We are responsible for all of the software on the factory floor, the warehouses, the financial systems, the restaurant, and even the public home page. Elon has called us the &quot;nervous system&quot; of SpaceX because we connect all of the other teams at SpaceX to ensure that the entire rocket building process runs smoothly.&lt;/p&gt;<br>&lt;p&gt;&lt;strong&gt;Responsibilities:&lt;/strong&gt;&lt;/p&gt;<br>&lt;ul&gt;<br>&lt;li&gt;We are seeking developers with demonstrable experience in: ASP.NET, C#, SQL Server, and AngularJS. We are a fast-paced, highly iterative team that has to adapt quickly as our factory grows. We need people who are comfortable tackling new problems, innovating solutions, and interacting with every facet of the company on a daily basis. Creative, motivated, able to take responsibility and support the applications you create. Help us get rockets out the door faster!&lt;/li&gt;<br>&lt;/ul&gt;<br>&lt;p&gt;&lt;strong&gt;Basic Qualifications:&lt;/strong&gt;&lt;/p&gt;<br>&lt;ul&gt;<br>&lt;li&gt;Bachelor&#39;s degree in computer science, engineering, physics, mathematics, or similar technical discipline.&lt;/li&gt;<br>&lt;li&gt;3+ years of experience developing across a full-stack:  Web server, relational database, and client-side (HTML/Javascript/CSS).&lt;/li&gt;<br>&lt;/ul&gt;<br>&lt;p&gt;&lt;strong&gt;Preferred Skills and Experience:&lt;/strong&gt;&lt;/p&gt;<br>&lt;ul&gt;<br>&lt;li&gt;Database - Understanding of SQL. Ability to write performant SQL. Ability to diagnose queries, and work with DBAs.&lt;/li&gt;<br>&lt;li&gt;Server - Knowledge of how web servers operate on a low-level. Web protocols. Designing APIs. How to scale web sites. Increase performance and diagnose problems.&lt;/li&gt;<br>&lt;li&gt;UI - Demonstrated ability creating rich web interfaces using a modern client side framework. Good judgment in UX/UI design.  Understands the finer points of HTML, CSS, and Javascript - know which tools to use when and why.&lt;/li&gt;<br>&lt;li&gt;System architecture - Knowledge of how to structure a database, web site, and rich client side application from scratch.&lt;/li&gt;<br>&lt;li&gt;Quality - Demonstrated usage of different testing patterns, continuous integration processes, build deployment systems. Continuous monitoring.&lt;/li&gt;<br>&lt;li&gt;Current - Up to date with current trends, patterns, goings on in the world of web development as it changes rapidly. Strong knowledge of computer science fundamentals and applying them in the real-world.&lt;/li&gt;<br>&lt;/ul&gt; &lt;br/&gt;&lt;br/&gt;&lt;/body&gt;&lt;/html&gt;</pre><ol><li>You need to go through this and remove all the HTML tags so that you’re only left with the text of the description. This is what you’ll then tokenize. Fortunately, throwing out all the HTML tags is easy with BeautifulSoup:</li></ol><pre>just_text = desc_bs.find_all(text=True)<br>print(just_text)</pre><pre>[&#39;About this job&#39;, &#39;\n&#39;, &#39;Location options: &#39;, &#39;Paid relocation&#39;, &#39;Job type: &#39;, &#39;Permanent&#39;, &#39;Experience level: &#39;, &#39;Mid-Level, Senior&#39;, &#39;Role: &#39;, &#39;Full Stack Developer&#39;, &#39;Industry: &#39;, &#39;Aerospace, Information Technology, Web Development&#39;, &#39;Company size: &#39;, &#39;1k-5k people&#39;, &#39;Company type: &#39;, &#39;Private&#39;, &#39;Technologies&#39;, &#39; &#39;, &#39;c#, sql, javascript, asp.net, angularjs&#39;, &#39; &#39;, &#39;Job description&#39;, &#39; &#39;, &#39;Full Stack Enterprise\xa0Software Engineer&#39;, &#39;\n&#39;, &#39;The EIS (Enterprise Information Systems) team writes the software that builds rockets and powers SpaceX. We are responsible for all of the software on the factory floor, the warehouses, the financial systems, the restaurant, and even the public home page. Elon has called us the &quot;nervous system&quot; of SpaceX because we connect all of the other teams at SpaceX to ensure that the entire rocket building process runs smoothly.&#39;, &#39;\n&#39;, &#39;Responsibilities:&#39;, &#39;\n&#39;, &#39;\n&#39;, &#39;We are seeking developers with demonstrable experience in: ASP.NET, C#, SQL Server, and AngularJS. We are a fast-paced, highly iterative team that has to adapt quickly as our factory grows. We need people who are comfortable tackling new problems, innovating solutions, and interacting with every facet of the company on a daily basis. Creative, motivated, able to take responsibility and support the applications you create. Help us get rockets out the door faster!&#39;, &#39;\n&#39;, &#39;\n&#39;, &#39;Basic Qualifications:&#39;, &#39;\n&#39;, &#39;\n&#39;, &quot;Bachelor&#39;s degree in computer science, engineering, physics, mathematics, or similar technical discipline.&quot;, &#39;\n&#39;, &#39;3+ years of experience developing across a full-stack:\xa0 Web server, relational database, and client-side (HTML/Javascript/CSS).&#39;, &#39;\n&#39;, &#39;\n&#39;, &#39;Preferred Skills and Experience:&#39;, &#39;\n&#39;, &#39;\n&#39;, &#39;Database - Understanding of SQL. Ability to write performant SQL. Ability to diagnose queries, and work with DBAs.&#39;, &#39;\n&#39;, &#39;Server - Knowledge of how web servers operate on a low-level. Web protocols. Designing APIs. How to scale web sites. Increase performance and diagnose problems.&#39;, &#39;\n&#39;, &#39;UI - Demonstrated ability creating rich web interfaces using a modern client side framework. Good judgment in UX/UI design.\xa0 Understands the finer points of HTML, CSS, and Javascript - know which tools to use when and why.&#39;, &#39;\n&#39;, &#39;System architecture - Knowledge of how to structure a database, web site, and rich client side application from scratch.&#39;, &#39;\n&#39;, &#39;Quality - Demonstrated usage of different testing patterns, continuous integration processes, build deployment systems. Continuous monitoring.&#39;, &#39;\n&#39;, &#39;Current - Up to date with current trends, patterns, goings on in the world of web development as it changes rapidly. Strong knowledge of computer science fundamentals and applying them in the real-world.&#39;, &#39;\n&#39;, &#39; &#39;]</pre><p>Superb; this is already broken down into what can be considered sentences!</p><ol><li>Join all these together, tokenize them, get rid of stop words, and then apply common tech job 2-grams:</li></ol><pre>joined = &#39; &#39;.join(just_text)<br>tokens = word_tokenize(joined)</pre><pre>stop_list = stopwords.words(&#39;english&#39;)<br>with_no_stops = [word for word in tokens if word not in stop_list]<br>cleaned = remove_punctuation(two_grammed)<br>print(cleaned)</pre><p>This yields the following output:</p><pre>[&#39;job&#39;, &#39;Location&#39;, &#39;options&#39;, &#39;Paid relocation&#39;, &#39;Job&#39;, &#39;type&#39;, &#39;Permanent&#39;, &#39;Experience&#39;, &#39;level&#39;, &#39;Mid-Level&#39;, &#39;Senior&#39;, &#39;Role&#39;, &#39;Full-Stack&#39;, &#39;Developer&#39;, &#39;Industry&#39;, &#39;Aerospace&#39;, &#39;Information Technology&#39;, &#39;Web Development&#39;, &#39;Company&#39;, &#39;size&#39;, &#39;1k-5k&#39;, &#39;people&#39;, &#39;Company&#39;, &#39;type&#39;, &#39;Private&#39;, &#39;Technologies&#39;, &#39;c#&#39;, &#39;sql&#39;, &#39;javascript&#39;, &#39;asp.net&#39;, &#39;angularjs&#39;, &#39;Job&#39;, &#39;description&#39;, &#39;Full-Stack&#39;, &#39;Enterprise Software&#39;, &#39;Engineer&#39;, &#39;EIS&#39;, &#39;Enterprise&#39;, &#39;Information&#39;, &#39;Systems&#39;, &#39;team&#39;, &#39;writes&#39;, &#39;software&#39;, &#39;builds&#39;, &#39;rockets&#39;, &#39;powers&#39;, &#39;SpaceX&#39;, &#39;responsible&#39;, &#39;software&#39;, &#39;factory&#39;, &#39;floor&#39;, &#39;warehouses&#39;, &#39;financial&#39;, &#39;systems&#39;, &#39;restaurant&#39;, &#39;even&#39;, &#39;public&#39;, &#39;home&#39;, &#39;page&#39;, &#39;Elon&#39;, &#39;called&#39;, &#39;us&#39;, &#39;nervous&#39;, &#39;system&#39;, &#39;SpaceX&#39;, &#39;connect&#39;, &#39;teams&#39;, &#39;SpaceX&#39;, &#39;ensure&#39;, &#39;entire&#39;, &#39;rocket&#39;, &#39;building&#39;, &#39;process&#39;, &#39;runs&#39;, &#39;smoothly&#39;, &#39;Responsibilities&#39;, &#39;seeking&#39;, &#39;developers&#39;, &#39;demonstrable experience&#39;, &#39;ASP.NET&#39;, &#39;C#&#39;, &#39;SQL Server&#39;, &#39;AngularJS&#39;, &#39;fast-paced&#39;, &#39;highly iterative&#39;, &#39;team&#39;, &#39;adapt quickly&#39;, &#39;factory&#39;, &#39;grows&#39;, &#39;need&#39;, &#39;people&#39;, &#39;comfortable&#39;, &#39;tackling&#39;, &#39;new&#39;, &#39;problems&#39;, &#39;innovating&#39;, &#39;solutions&#39;, &#39;interacting&#39;, &#39;every&#39;, &#39;facet&#39;, &#39;company&#39;, &#39;daily&#39;, &#39;basis&#39;, &#39;Creative&#39;, &#39;motivated&#39;, &#39;able&#39;, &#39;take&#39;, &#39;responsibility&#39;, &#39;support&#39;, &#39;applications&#39;, &#39;create&#39;, &#39;Help&#39;, &#39;us&#39;, &#39;get&#39;, &#39;rockets&#39;, &#39;door&#39;, &#39;faster&#39;, &#39;Basic&#39;, &#39;Qualifications&#39;, &#39;Bachelor&#39;, &quot;&#39;s&quot;, &#39;degree&#39;, &#39;computer science&#39;, &#39;engineering&#39;, &#39;physics&#39;, &#39;mathematics&#39;, &#39;similar&#39;, &#39;technical&#39;, &#39;discipline&#39;, &#39;3+&#39;, &#39;years&#39;, &#39;experience&#39;, &#39;developing&#39;, &#39;across&#39;, &#39;full-stack&#39;, &#39;Web server&#39;, &#39;relational database&#39;, &#39;client-side&#39;, &#39;HTML/Javascript/CSS&#39;, &#39;Preferred&#39;, &#39;Skills&#39;, &#39;Experience&#39;, &#39;Database&#39;, &#39;Understanding&#39;, &#39;SQL&#39;, &#39;Ability&#39;, &#39;write&#39;, &#39;performant&#39;, &#39;SQL&#39;, &#39;Ability&#39;, &#39;diagnose&#39;, &#39;queries&#39;, &#39;work&#39;, &#39;DBAs&#39;, &#39;Server&#39;, &#39;Knowledge&#39;, &#39;web&#39;, &#39;servers&#39;, &#39;operate&#39;, &#39;low-level&#39;, &#39;Web&#39;, &#39;protocols&#39;, &#39;Designing&#39;, &#39;APIs&#39;, &#39;scale&#39;, &#39;web&#39;, &#39;sites&#39;, &#39;Increase&#39;, &#39;performance&#39;, &#39;diagnose&#39;, &#39;problems&#39;, &#39;UI&#39;, &#39;Demonstrated&#39;, &#39;ability&#39;, &#39;creating&#39;, &#39;rich&#39;, &#39;web&#39;, &#39;interfaces&#39;, &#39;using&#39;, &#39;modern&#39;, &#39;client-side&#39;, &#39;framework&#39;, &#39;Good&#39;, &#39;judgment&#39;, &#39;UX/UI&#39;, &#39;design&#39;, &#39;Understands&#39;, &#39;finer&#39;, &#39;points&#39;, &#39;HTML&#39;, &#39;CSS&#39;, &#39;Javascript&#39;, &#39;know&#39;, &#39;tools&#39;, &#39;use&#39;, &#39;System&#39;, &#39;architecture&#39;, &#39;Knowledge&#39;, &#39;structure&#39;, &#39;database&#39;, &#39;web&#39;, &#39;site&#39;, &#39;rich&#39;, &#39;client-side&#39;, &#39;application&#39;, &#39;scratch&#39;, &#39;Quality&#39;, &#39;Demonstrated&#39;, &#39;usage&#39;, &#39;different&#39;, &#39;testing&#39;, &#39;patterns&#39;, &#39;continuous integration&#39;, &#39;processes&#39;, &#39;build&#39;, &#39;deployment&#39;, &#39;systems&#39;, &#39;Continuous monitoring&#39;, &#39;Current&#39;, &#39;date&#39;, &#39;current trends&#39;, &#39;patterns&#39;, &#39;goings&#39;, &#39;world&#39;, &#39;web development&#39;, &#39;changes&#39;, &#39;rapidly&#39;, &#39;Strong&#39;, &#39;knowledge&#39;, &#39;computer science&#39;, &#39;fundamentals&#39;, &#39;applying&#39;, &#39;real-world&#39;]</pre><p>You now have a very nice and refined set of keywords pulled out of the job listing.</p><p>If you enjoyed this article and want to learn more about several other web scraping concepts, you can always refer to this book, <a href="https://www.amazon.com/Python-Web-Scraping-Cookbook-microservices-ebook/dp/B077NC4TQP">Python Web Scraping Cookbook</a>. With over 90 proven recipes to get you scraping with Python, microservices, Docker, and AWS, the book is a must-have for Python programmers, web administrators, security professionals, and anyone interested in performing web analytics.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4c7323061ae5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[3 tricks for efficient data science]]></title>
            <link>https://medium.com/@pablito_prague/3-tricks-for-efficient-data-science-876a80e2b0f9?source=rss-17ce4d4503b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/876a80e2b0f9</guid>
            <dc:creator><![CDATA[Pablo Maldonado]]></dc:creator>
            <pubDate>Sun, 22 Apr 2018 13:07:56 GMT</pubDate>
            <atom:updated>2018-04-22T13:08:00.429Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s amazing how little things can turn a data science / mathematical modelling project into a full-fledged mess. There are easily avoidable traps, but unfortunately also easily forgotten: we love to pretend we “couldn’t do something that silly”.</p><p>Following the principles below has saved me (and others) a lot of time and spared me from unnecessary pain:</p><h4>Always check your data</h4><p>No one has ever been arrested for excess logging. If you don’t check the input for your models <em>and</em> the outputs of calculations at the intermediate steps, you are in for disappointing results.</p><p>There is a promise of a greener future when deep learning will release us from all the mess of pre-processing, and we can ingest raw data in any format and the intermediate layers will sort it out. Sorry, but here on real life we are still too far from that. Garbage in, garbage out.</p><h4>Test your model on limit cases</h4><p>This is somewhat related to the previous point, but not always. What happens to your model in the limit? Many scientists ask these kind of questions as a basic sanity check. The same should be done: does your model predict what the domain expert / common sense expects when you make one variable too large or too small? If not, then there is something wrong. Do you have a positive regression coefficient for monthly totals, but a negative for average totals? Time to revisit your model.</p><p>A common source of errors here is the scale: having two variables on wildly different scales often produces odd, counter-intuitive models.</p><h4>Fail fast</h4><p>A common error, specially in highly technical teams, is to over-engineer prematurely. There is a justified fear from that: if the data science experiments become too messy, then it will be too complicated for the engineers to implement them when it’s time to scale. But very rarely this justifies discarding the working Python prototype for writing everything from scratch in C.</p><p>Another common error is to try the “tool of the day” and build the solution in a poorly supported technology. There are thousands of poorly maintained and documented open source projects.</p><p>Don’t reinvent the wheel. Bet on tested technologies to iterate quickly and, ultimately, bring value to your final user, whether a consulting client, or someone else internally in your organization. Once the project is on its way and there’s trust on the final user side, there will be plenty of time to explore alternatives.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=876a80e2b0f9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cross-entropy method in R]]></title>
            <link>https://medium.com/@pablito_prague/cross-entropy-method-in-r-754c8048eaae?source=rss-17ce4d4503b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/754c8048eaae</guid>
            <dc:creator><![CDATA[Pablo Maldonado]]></dc:creator>
            <pubDate>Mon, 18 Dec 2017 13:14:16 GMT</pubDate>
            <atom:updated>2017-12-18T13:14:19.955Z</atom:updated>
            <content:encoded><![CDATA[<p>The cross entropy method is a general approach to optimization that relies in two nice ideas. In the context of finding the maximum of a scalar-valued function this means:</p><ol><li>Generate some random parameters and evaluate the function there.</li><li>Use the best values of the parameter to generate new candidates.</li></ol><p>One simple way of generating such random parameters is fitting a normal distribution at every iteration: we choose a subset of “elite parameters” (a fraction of those we tried), calculate their mean and covariance and use it to generate a new population of parameters.</p><p>This kind of optimization methods is quite useful for instance in reinforcement learning.</p><p>For instance, suppose we want to maximize the function:</p><pre># the function we want to optimize<br>f &lt;-  function(theta){<br>  reward = -sum((solution - theta)**2)<br>  return(reward)<br>}<br><br>  solution &lt;- c(0.5, 0.1, -0.3)</pre><p>Then the cross-entropy algorithm in this case is:</p><pre>library(MASS)<br><br>cem &lt;- function(f, n_iter, theta_mean, theta_std, batch_size=25, elite_frac=0.2){<br>  # Now, for the algorithms<br>  for(it in 1:n_iter){<br>    # Sample parameter vectors<br>    thetas &lt;-  matrix(mvrnorm(n=batch_size*dim_theta, mu= theta_mean, Sigma=theta_std)<br>, ncol = dim_theta)<br>    rewards &lt;- apply(thetas,1,f) <br>    # Get elite parameters<br>    n_elite &lt;-  as.integer(batch_size * elite_frac)<br>    elite_inds &lt;-  sort(rewards, decreasing = T, index.return=T)$ix[1:n_elite]<br>    elite_thetas &lt;- thetas[elite_inds,]<br>    # Update theta_mean, theta_std<br>    theta_mean &lt;- apply(elite_thetas, 2,mean)<br>    theta_std &lt;- 0.01*diag(dim_theta)+0.99*cov(elite_thetas)<br>  }<br> return(theta_mean)<br>}  <br><br>cem(f,300)</pre><p>and we call this like:</p><pre>dim_theta &lt;-  length(solution)<br>  theta_mean &lt;-  matrix(0,dim_theta,1)<br>  theta_std &lt;-  diag(dim_theta)<br>  batch_size &lt;-  25 # number of samples per batch<br>  elite_frac &lt;-  0.2 # fraction of samples used as elite set<br>  <br>cem(f,300, theta_mean=theta_mean, theta_std=theta_std<br>, batch_size=batch_size, elite_frac=elite_frac)</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=754c8048eaae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Las penas con Python son menos]]></title>
            <link>https://medium.com/@pablito_prague/las-penas-con-python-son-menos-a82da58fc3c5?source=rss-17ce4d4503b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/a82da58fc3c5</guid>
            <dc:creator><![CDATA[Pablo Maldonado]]></dc:creator>
            <pubDate>Thu, 07 Sep 2017 15:27:34 GMT</pubDate>
            <atom:updated>2017-09-08T09:07:13.981Z</atom:updated>
            <content:encoded><![CDATA[<p>No, no es un post de terapia. El post es sobre un algoritmo de teoría de juegos (regret matching) en Python.</p><p>Desde mi doctorado (teoría de juegos, pero pesado en el análisis matemático y las ecuaciones diferenciales parciales) me interesé en algoritmos agradables y sencillos que superarían las construcciones teóricas de fantasía. Uno de ellos pasa a ser la minimización del lamento.</p><p>Supongamos que queremos jugar piedra, papel, tijeras por dinero. Cada uno de nosotros pone 100 euros en la mesa, jugamos y el ganador se lleva el dinero. Supongamos además que juegas piedra y juego papel (así que gano) y así tomar los 100 euros de la mesa. En este caso, tu arrepentimiento por no jugar papel es de 100 (porque habríamos atado), pero tu arrepentimiento por no pagar tijeras es 200, porque no ganó.</p><p>Formalmente, si $$ I $$, $$ J $$ denotan los conjuntos de estrategias de los jugadores (aquí $$ I = J $$ y ambos son iguales a {piedra, papel, tijeras}), entonces el arrepentimiento para el jugador 1 acción $$ i $$, dado los movimientos de última ronda $$ i ^ *, j ^ * $$ es:</p><p>$$ \ max (u (i, j ^ *) — u (i ^ *, j ^ *), 0). $$</p><p>¿Cómo conseguimos una estrategia de esto? Inicializamos un contador al principio del juego. Jugamos al azar en el principio, y desde la segunda ronda, calculamos los arrepentimientos por cada acción y dividimos cada uno de estos números por la suma de los lamentos de nuestras acciones. Elegiremos la acción con una estrategia aleatoria, jugando con probabilidad proporcional al arrepentimiento acumulativo normalizado.</p><p>Código en Python a continuación:</p><pre>import matplotlib.pyplot as plt<br>import random<br>import numpy as np<br><br>np.set_printoptions(precision=2)<br><br># Base player that chooses a constant mixed strategy every time<br>class Player:<br>    def __init__(self):<br>        self.my_moves = []<br>        self.other_moves = []<br><br>    def move(self, strategy):<br>        # Input: a vector of probability distributions for actions<br>        # Output: a pure action<br><br>        r = random.uniform(0,1)<br>        n_actions = len(strategy)<br><br>        a = 0<br>        cumulative_proba = 0.0<br><br>        while a&lt;n_actions-1:<br>            cumulative_proba += strategy[a] <br>            if r &lt; cumulative_proba: return a<br>            a +=1<br>        return a<br><br>class RegretPlayer(Player):<br>    def __init__(self):<br>        super(RegretPlayer, self).__init__()<br>        self.regret_sum = np.zeros(3)<br><br>    def regret(self):<br><br>        if len(self.my_moves)&gt;0:<br>            my_action = self.my_moves[-1]<br>            his_action = self.other_moves[-1]<br>        else:<br>            return np.zeros(3)<br><br>        # Payoffs from my perspective<br>        my_payoff = np.zeros(3)<br><br>        # If we play the same, I don&#39;t get any payoff<br>        my_payoff[his_action] = 0.<br><br>        # I win when he plays scissors and I pay rock, <br>        # or when I play the &quot;next&quot; (Rock = 0, Paper = 1, Scissors = 2)<br>        my_payoff[0 if his_action == 2 else his_action + 1] = 1<br><br>        # I lose when I play scissors and he plays rock, <br>        # or when I play the &quot;previous&quot; action         <br>        my_payoff[2 if his_action == 0 else his_action -1] = -1<br><br>        regrets = [my_payoff[a]-my_payoff[my_action] for a in range(3)]<br>        regrets = np.array(regrets)<br>        return regrets<br><br>    def get_regret_mixed_strategy(self):<br><br>        normalize_sum = 0.0<br>        strategy = np.zeros(3)<br>        regret = self.regret()                   <br><br>        for a in range(3):<br>            strategy[a] = max(self.regret_sum[a],0)<br>            normalize_sum += strategy[a]<br><br>        # If all regrets are positive, play randomly<br>        if normalize_sum &gt; 0:<br>            strategy = strategy / normalize_sum<br>        else:<br>            strategy = np.ones(3)/3<br><br>        self.regret_sum += regret<br><br>        return strategy            <br><br>def payoff(my_action, his_action):<br>    if my_action == his_action: <br>        return 0<br>    if his_action == 0 and my_action==2 or my_action == his_action-1:<br>        return -1<br>    return 1<br><br>def run(n_rounds = 10):<br>    p1 = RegretPlayer()<br>    p2 = Player() <br><br>    total_p1 = 0.0<br>    total_p2 = 0.0<br><br>    rounds_won_p1 = 0.<br><br>    plt.ion()<br>    plt.axis([-0.1,n_rounds+0.1,-0.1,1.1])<br>    print(&quot;*&quot;*100)<br>    print(&quot;The match begins&quot;)<br>    print(&quot;*&quot;*100)<br>    for n in range(1,n_rounds):<br><br>        regret_strategy_p1 = p1.get_regret_mixed_strategy()<br><br>        m1 = p1.move(regret_strategy_p1)<br><br>        m2 = p2.move([0.4,0.3,0.3])<br><br>        # Players update the info of the moves<br>        p1.my_moves.append(m1)<br>        p1.other_moves.append(m2)    <br><br>        total_p1 += payoff(m1,m2)<br><br><br>        #### SHOW RESULTS<br>        moves_map = {0:&quot;Rock&quot;, 1:&quot;Paper&quot;, 2:&quot;Scissors&quot;}<br>        print(&#39;-&#39;*50)<br>        print(&quot;Regret:&quot;)<br>        print(p1.regret_sum)<br>        print(&quot;Strategy:&quot;, regret_strategy_p1)<br>        print(&quot;My move: %s&quot; % moves_map[m1])<br>        print(&quot;His move: %s&quot; % moves_map[m2])<br>        print(&quot;Payoffs:&quot;)<br>        print(total_p1)<br>        print(total_p2)<br><br>        rounds_won_p1 += 1 if payoff(m1,m2) &gt;0 else 0<br><br>        # Plot the moves<br>        plt.title(&quot;Percentage of rounds won using a regret strategy&quot;)<br>        plt.scatter(n,rounds_won_p1/n, color = &quot;red&quot;)<br><br>        plt.show()<br>        plt.pause(0.1)<br><br>if __name__ == &quot;__main__&quot;:<br>    run(n_rounds = 100)</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a82da58fc3c5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Regret minimization in Python]]></title>
            <link>https://medium.com/@pablito_prague/regret-minimization-in-python-46fd2e703c30?source=rss-17ce4d4503b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/46fd2e703c30</guid>
            <dc:creator><![CDATA[Pablo Maldonado]]></dc:creator>
            <pubDate>Thu, 07 Sep 2017 15:21:36 GMT</pubDate>
            <atom:updated>2017-09-08T09:06:53.186Z</atom:updated>
            <content:encoded><![CDATA[<p>Since my PhD (game theory, but heavy in mathematical analysis and partial differential equations) I was interested in nice, simple algorithms that would beat fancy theoretical constructs. One of those happens to be regret minimization.</p><p>Suppose we want to play rock, paper, scissors for money. Each of us puts 100 EUR on the table, we play and the winner takes the money. Let’s further assume that you play rock and I play paper (so I win) and thus take the 100 EUR from the table. In this case, your <strong>regret </strong>for not playing paper is 100 (because we would have tied), but your regret for not paying scissors is 200, because you did not win.</p><p>Formally, if $$I$$, $$J$$ denote the strategy sets of the players (here $$I=J$$ and both are equal to {rock, paper, scissors}), then the regret for player 1’s action $$i$$, given the last-round moves $$i^*, j^*$$ is:</p><p>$$ \max(u(i,j^*)-u(i^*,j^*),0).$$</p><p>How do we get a strategy out of this? We initialize a counter at the beginning of the game. We play randomly in the beginning, and from the second round on, we compute the regrets for each action and divide each of these numbers by the sum of regrets of our actions. We will choose the action with a random strategy, playing with probability proportional to the normalized cumulative regret.</p><p>Code in Python below:</p><pre>import matplotlib.pyplot as plt<br>import random<br>import numpy as np<br><br>np.set_printoptions(precision=2)<br><br># Base player that chooses a constant mixed strategy every time<br>class Player:<br>    def __init__(self):<br>        self.my_moves = []<br>        self.other_moves = []<br><br>    def move(self, strategy):<br>        # Input: a vector of probability distributions for actions<br>        # Output: a pure action<br><br>        r = random.uniform(0,1)<br>        n_actions = len(strategy)<br><br>        a = 0<br>        cumulative_proba = 0.0<br><br>        while a&lt;n_actions-1:<br>            cumulative_proba += strategy[a] <br>            if r &lt; cumulative_proba: return a<br>            a +=1<br>        return a<br><br>class RegretPlayer(Player):<br>    def __init__(self):<br>        super(RegretPlayer, self).__init__()<br>        self.regret_sum = np.zeros(3)<br><br>    def regret(self):<br><br>        if len(self.my_moves)&gt;0:<br>            my_action = self.my_moves[-1]<br>            his_action = self.other_moves[-1]<br>        else:<br>            return np.zeros(3)<br><br>        # Payoffs from my perspective<br>        my_payoff = np.zeros(3)<br><br>        # If we play the same, I don&#39;t get any payoff<br>        my_payoff[his_action] = 0.<br><br>        # I win when he plays scissors and I pay rock, <br>        # or when I play the &quot;next&quot; (Rock = 0, Paper = 1, Scissors = 2)<br>        my_payoff[0 if his_action == 2 else his_action + 1] = 1<br><br>        # I lose when I play scissors and he plays rock, <br>        # or when I play the &quot;previous&quot; action         <br>        my_payoff[2 if his_action == 0 else his_action -1] = -1<br><br>        regrets = [my_payoff[a]-my_payoff[my_action] for a in range(3)]<br>        regrets = np.array(regrets)<br>        return regrets<br><br>    def get_regret_mixed_strategy(self):<br><br>        normalize_sum = 0.0<br>        strategy = np.zeros(3)<br>        regret = self.regret()                   <br><br>        for a in range(3):<br>            strategy[a] = max(self.regret_sum[a],0)<br>            normalize_sum += strategy[a]<br><br>        # If all regrets are positive, play randomly<br>        if normalize_sum &gt; 0:<br>            strategy = strategy / normalize_sum<br>        else:<br>            strategy = np.ones(3)/3<br><br>        self.regret_sum += regret<br><br>        return strategy            <br><br>def payoff(my_action, his_action):<br>    if my_action == his_action: <br>        return 0<br>    if his_action == 0 and my_action==2 or my_action == his_action-1:<br>        return -1<br>    return 1<br><br>def run(n_rounds = 10):<br>    p1 = RegretPlayer()<br>    p2 = Player() <br><br>    total_p1 = 0.0<br>    total_p2 = 0.0<br><br>    rounds_won_p1 = 0.<br><br>    plt.ion()<br>    plt.axis([-0.1,n_rounds+0.1,-0.1,1.1])<br>    print(&quot;*&quot;*100)<br>    print(&quot;The match begins&quot;)<br>    print(&quot;*&quot;*100)<br>    for n in range(1,n_rounds):<br><br>        regret_strategy_p1 = p1.get_regret_mixed_strategy()<br><br>        m1 = p1.move(regret_strategy_p1)<br><br>        m2 = p2.move([0.4,0.3,0.3])<br><br>        # Players update the info of the moves<br>        p1.my_moves.append(m1)<br>        p1.other_moves.append(m2)    <br><br>        total_p1 += payoff(m1,m2)<br><br><br>        #### SHOW RESULTS<br>        moves_map = {0:&quot;Rock&quot;, 1:&quot;Paper&quot;, 2:&quot;Scissors&quot;}<br>        print(&#39;-&#39;*50)<br>        print(&quot;Regret:&quot;)<br>        print(p1.regret_sum)<br>        print(&quot;Strategy:&quot;, regret_strategy_p1)<br>        print(&quot;My move: %s&quot; % moves_map[m1])<br>        print(&quot;His move: %s&quot; % moves_map[m2])<br>        print(&quot;Payoffs:&quot;)<br>        print(total_p1)<br>        print(total_p2)<br><br>        rounds_won_p1 += 1 if payoff(m1,m2) &gt;0 else 0<br><br>        # Plot the moves<br>        plt.title(&quot;Percentage of rounds won using a regret strategy&quot;)<br>        plt.scatter(n,rounds_won_p1/n, color = &quot;red&quot;)<br><br>        plt.show()<br>        plt.pause(0.1)<br><br>if __name__ == &quot;__main__&quot;:<br>    run(n_rounds = 100)</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=46fd2e703c30" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is it really worth learning AI/ML?]]></title>
            <link>https://medium.com/@pablito_prague/is-it-really-worth-learning-ai-ml-b3016b5479f?source=rss-17ce4d4503b1------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3016b5479f</guid>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Pablo Maldonado]]></dc:creator>
            <pubDate>Sun, 20 Aug 2017 21:23:50 GMT</pubDate>
            <atom:updated>2018-01-04T09:45:08.005Z</atom:updated>
            <content:encoded><![CDATA[<p>Every minute, a new data science course is born somewhere. Granted, I totally made this up but I might not be completely wrong. Lots of research groups are doing breakthroughs in data science and artificial intelligence everyday, and yes, there is hype, but there is also optimism, strongly supported by evidence.</p><p>So, is it really time to jump in and drop one’s career and everything to switch to AI? Is it finally time to join Coursera/Udacity/Edx/your favorite MOOC/university?</p><p>Well, no.</p><p>The thing is, AI and ML, hype or not, are tools. They are hammers waiting for nails.</p><blockquote>It doesn’t make much sense to invest in machine learning training by itself. Successful applications of machine learning and, in general, artificial intelligence, often require solid domain knowledge and proprietary data, one way or another.</blockquote><p>Knowledge is certainly transferable among machine learning applications, and solving one type of problems (yes, even mildly artificial, like Kaggle) does help to solve unseen problems, and you can get lucky and apply out of the box algorithms with default parameters and match the state-of-the-art in a particular domain, or exceed it. But going beyond the easy-hanging fruit takes tons of effort and a combination of the above.</p><p>A better career strategy might be choosing a domain one is interested in, and then see how AI can be applied there. And for this, you just need to skim through some of the excellent courses around, no need to get a PhD in machine learning.</p><figure><img alt="Coursera AH Purple Design 2" src="https://cdn-images-1.medium.com/max/300/0*ZYSZxhb9eDmVHR2a." /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3016b5479f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>