<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Alex Browne on Medium]]></title>
        <description><![CDATA[Stories by Alex Browne on Medium]]></description>
        <link>https://medium.com/@albrow?source=rss-6db1be83348f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*eLXdcczgLiMd62ULt3_b7g.jpeg</url>
            <title>Stories by Alex Browne on Medium</title>
            <link>https://medium.com/@albrow?source=rss-6db1be83348f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 09:35:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@albrow/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Using Stable Diffusion to Improve Image Segmentation Models]]></title>
            <link>https://medium.com/edge-analytics/using-stable-diffusion-to-improve-image-segmentation-models-1e99c25acbf?source=rss-6db1be83348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e99c25acbf</guid>
            <category><![CDATA[stable-diffusion]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <category><![CDATA[computer-vision]]></category>
            <dc:creator><![CDATA[Alex Browne]]></dc:creator>
            <pubDate>Thu, 12 Jan 2023 20:02:06 GMT</pubDate>
            <atom:updated>2023-01-12T20:02:06.500Z</atom:updated>
            <content:encoded><![CDATA[<p>As ML engineers, the teams at <a href="https://edgeanalytics.io/">Edge Analytics</a> and <a href="https://infinity.ai/">Infinity AI</a> are very familiar with the challenges associated with obtaining high quality labeled images for computer vision applications. With the release of generative image models, such as open-source <a href="https://huggingface.co/spaces/stabilityai/stable-diffusion">Stable Diffusion</a> from <a href="https://stability.ai/">Stability AI</a>, we explored the use of generative models to improve performance on a specific semantic segmentation model.</p><p><a href="https://stability.ai/blog/stable-diffusion-public-release">Stable Diffusion</a> is an impressively powerful text-to-image model released by Stability AI earlier this year. In this blog post, we’ll explore a technique for augmenting training data with Stable Diffusion in order to improve performance on an image segmentation task. This approach is <em>especially </em>powerful in applications where data is limited or would otherwise require tedious human labeling.</p><p>In the context of computer vision models, image segmentation refers to splitting an image into two or more components based on its contents. In contrast to “image classification”, the goal of segmentation is to not only identify <em>what</em> an image contains, but <em>which</em> <em>parts</em> of the image correspond to each class.</p><p>Specifically, we will be taking a look at the <a href="https://www.kaggle.com/datasets/balraj98/deepglobe-road-extraction-dataset">DeepGlobe Road Extraction Dataset</a>, which consists of around 6,000 aerial photographs of rural roads. The task for this dataset is to separate images into two classes: “road” and “background”. The dataset also comes with training labels in the form of mask images, where roads are identified in white and background is in black.</p><figure><img alt="Example data from the DeepGlobe Road Extraction Dataset" src="https://cdn-images-1.medium.com/max/1024/1*75BktdzJtn-kCeEjntcvig.png" /><figcaption>Left: input image. Right: label with “roads” in white and “background” in black.</figcaption></figure><h3>Baseline Training Procedure</h3><p>Let’s jump into the nitty gritty details. We split the dataset into 70/15/15 train/test/validation, then, as a baseline, trained a basic U-Net model on the full training set of 4,358 images. We trained this model (and later models) for 50 epochs and restored the best weights based on val_loss after training. We won’t show the code here for loading the data and training the baseline model, just so we can get to the interesting part faster. However, the full code is available in <a href="https://colab.research.google.com/drive/1xmOO0tyPQDOEQ2zAFfk30hjND-moVqeR?usp=sharing">this Colab Notebook</a>.</p><p>We measured model performance against the held-out test set with two different metrics: binary cross-entropy (also used as the loss function during training) and <a href="https://learnopencv.com/intersection-over-union-iou-in-object-detection-and-segmentation/">intersection over union</a>, the latter of which was used to judge winners in the original <a href="https://competitions.codalab.org/competitions/18467#learn_the_details-evaluation">DeepGlobe Challenge</a>.</p><figure><img alt="Baseline model performance" src="https://cdn-images-1.medium.com/max/1024/1*VzMSkfo219LRYokBn0JTLA.png" /><figcaption>Baseline model using the full training dataset.</figcaption></figure><p>To improve our intuition around what these numbers really mean, let’s look at some examples of model predictions from the held-out test set. It seems the model is doing a pretty good job of finding the roads.</p><figure><img alt="Images showing the model predictions" src="https://cdn-images-1.medium.com/max/485/1*3dq68yqk4FifbSAqF55Tzw.png" /><figcaption>Model predictions for images in the test set.</figcaption></figure><h3>Simulating Limited Data</h3><p>Baseline performance is useful as a point of comparison, but what we’re really interested in for the purposes of this blog post is whether Stable Diffusion can help in data-limited applications — cases where we only have tens or hundreds of samples to train on. Suppose you didn’t have the time or money to capture and tediously label thousands of images. Could Stable Diffusion help you do more with less?</p><p>To help answer this question we’re going to simulate a limited dataset by training a model on just 100 images. Then we’ll augment that limited dataset by generating “image variants” with Stable Diffusion. To do that, we’ll first collect performance metrics for a new model trained on just 100 images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bNlGOgtZ11iIHq63WZjA_g.png" /><figcaption>Metrics for a model trained on a limited dataset of just 100 images.</figcaption></figure><p>Unsurprisingly, this extreme reduction in dataset size results in a regression on both performance metrics. If we look at some sample predictions, it looks like the model is not doing a very good job at all.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/485/1*l0HL7_LzVNpxLOap2VIuLw.png" /><figcaption>Example predictions for the data-limited model.</figcaption></figure><h3>Augmenting Data with Stable Diffusion</h3><p>Now, the fun part! Let’s use Stable Diffusion to generate “variants” for each of the 100 images in our limited training set. Unfortunately, in this case we can’t simply ask Stable Diffusion to generate images from scratch. That’s because for image segmentation training, we need <em>both</em> the input images <em>and</em> the corresponding labels/masks. While Stable Diffusion is a powerful model, it’s not capable of accurately generating both the input images and the mask images (and if it were, we wouldn’t need to train our own U-Net model in the first place!). Instead, we’re going to use a clever trick using the in-painting feature.</p><p>The <a href="https://huggingface.co/stabilityai/stable-diffusion-2-inpainting">in-painting feature</a> makes it so that Stable Diffusion only replaces or redraws <em>specific parts</em> of an image. For our use case, we’re going to use Stable Diffusion to create image “variants” by only redrawing the “background” part of the image. Since the “road” part of the image remains unchanged, we can re-use the existing mask images in the training dataset as labels! Here’s an example of what a generated image variant looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ecrv5dsemToauryvdy_0-A.png" /><figcaption>Left: image mask. Middle: original image. Right: image variant generated by Stable Diffusion.</figcaption></figure><p>Here’s the key piece of code we used to generate the image variants (remember, the full code is <a href="https://colab.research.google.com/drive/1xmOO0tyPQDOEQ2zAFfk30hjND-moVqeR?usp=sharing">available on Google Colab</a>). Note that we don’t want Stable Diffusion to generate new roads, because that would make the existing labels wrong. We only want it to generate background scenery. To dissuade the model from adding roads, we included “road” and “highway” in the negative prompt.</p><pre># Load and configure Stable Diffusion inpainting model<br>pipe = StableDiffusionInpaintPipeline.from_pretrained(<br>  &quot;runwayml/stable-diffusion-inpainting&quot;,<br>  revision=&quot;fp16&quot;,<br>  torch_dtype=torch.float16<br>)<br>pipe = pipe.to(&quot;cuda&quot;)<br><br># Create prompts for image generation.<br>BACKGROUND_PROMPT = &quot;Overhead aerial photograph of a rural area, 8K, high resolution, satellite image, extremely detailed&quot;<br># Note the inclusion of &quot;road&quot; and &quot;highway&quot; in the negative prompt. We don&#39;t<br># want Stable Diffusion to add any more roads to the image because we are not<br># changing the mask during training.<br>BACKGROUND_NEGATIVE_PROMPT = &quot;road, highway, fisheye&quot;<br><br>def create_image_variant(image_path, mask_path):<br>  # Use PIL to resize the images to 512x512, the optimal size for Stable<br>  # Diffusion.<br>  init_image = PIL.Image.open(image_path).convert(&quot;RGB&quot;).resize((512, 512))<br>  mask_image = PIL.Image.open(mask_path).convert(&quot;RGB&quot;).resize((512, 512))<br>  # Note we are inverting the mask in this case because we want to re-draw<br>  # everything that is *not* part of the road.<br>  inverted_mask = PIL.ImageOps.invert(mask_image)<br><br>  # Use Stable Diffusion to re-paint the background scenery.<br>  gen_background_image = pipe(<br>    prompt=BACKGROUND_PROMPT,<br>    negative_prompt=BACKGROUND_NEGATIVE_PROMPT,<br>    image=init_image,<br>    mask_image=inverted_mask<br>  ).images[0]<br>  <br>  # For some reason, this version of Stable Diffusion is not completely<br>  # respecting the image mask and will distort/repaint portions of the image<br>  # that it is not supposed to. We can workaround this by re-applying the mask<br>  # to the generated image.<br>  # See https://github.com/runwayml/stable-diffusion/issues/5<br>  final_image = PIL.Image.composite(<br>    gen_background_image,<br>    init_image,<br>    inverted_mask.convert(&quot;L&quot;)<br>  )<br><br>  return final_image</pre><p>That’s it! With just a few lines of code we can start augmenting our training dataset with generated images.</p><h3>Re-Training with Augmented Datasets</h3><p>With a plan in place, the next step was to train the model again using the new generated image variants. To test the limits of this approach, we tested with 1, 4, and 16 variants per source image and compared the results. Importantly, image variants were <em>only</em> generated from the 100 samples from the limited training set, giving us an accurate view of what might happen if we only had those 100 images to start with. We used the same training procedure as before.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6gsX94MBQx7brf5cLU72HQ.png" /></figure><p>Including the generated image variants measurably improved performance compared to only using the original 100 images. If we look at the example predictions for the 4-variant model, we see a definite improvement over the previous model. These predictions are still not perfect, but they are <em>much</em> better, especially considering we started with the same set of just 100 images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/484/1*c5DX9UBOOJ_byhG5f8rO3A.png" /><figcaption>Example predictions for the 4-variant model.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/485/1*l0HL7_LzVNpxLOap2VIuLw.png" /><figcaption>For comparison: example predictions for the previous data-limited model.</figcaption></figure><h3>Limitations</h3><p>Generally speaking, including more variants per image increased performance. However, it seems we are running up against the limits of this approach when generating 16 variants per source image. While the 16-variant model has the best IoU metric, we saw degradation in binary cross-entropy (even worse than the model without any augmented data). Additionally, the loss curves for the 16-variant model show evidence of over-fitting during training. Keep in mind that we didn’t change the “road” part of the image at all for any of the image variants. Given this, one explanation is that the model is over-fitting on either the shape of the road, the individual pixels that make up the road, or both.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*s555N5LJ3mhnTjG0c_7Yxg.png" /><figcaption>Loss curves during training of the 16-variant model.</figcaption></figure><p>Of course, none of our augmented datasets were good enough to surpass the baseline model trained on the full dataset of thousands of images. That was never really our goal in the first place and is not terribly surprising. The reality is that not everyone will have the time or money required to capture and label thousands of images. What we’ve shown here is that you can use generated data to <em>meaningfully</em> improve model performance in data-limited applications.</p><h3>Possible Improvements</h3><p>There are a number of traditional techniques we could use that might improve model performance, including more basic image augmentation (flipping, zooming, and rotating), hyperparameter optimization, and fine-tuning a larger pre-trained model. These, however, are out of scope of this blog, so we’ll focus on generative image improvements only.</p><p>The biggest problem with the approach described here is that because we were not changing the “road” part of the image between variants, the model eventually starts to over-fit. There are a few different ways that this could be addressed:</p><ol><li>Use Stable Diffusion to redraw the “road” part of the image as well as the “background” part. We tried doing this, but Stable Diffusion had a hard time generating roads in many cases, possibly because some of the lines in the mask image are very thin (sometimes less than 1 pixel thick after resizing). For other use cases that do not have thin masks, this may be a much easier thing to do.</li><li>Break each original image into multiple parts, generate variants for each part, then stitch the different parts together to create new road shapes. For example, you could take the bottom half of one source image and combine it with the top half of a different source image. This would allow us to use different road shapes during training while leveraging Stable Diffusion to hide the seams and generate a final image that looks cohesive.</li><li>Instead of using the existing mask images, generate new ones from scratch procedurally (e.g. using a simple script to draw white lines on a black background with different patterns), then use Stable Diffusion to fill in both the “road” and the “background”. We didn’t do this in our blog post because of the time it would take to write the necessary code (remember, the whole point was to <em>reduce</em> the time and effort you need to generate good training data). But for some use-cases, the up-front cost of writing the additional code might be worth it.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YhPA6UUOV0Qu8t9-" /></figure><p><a href="https://edgeanalytics.io/">Edge Analytics</a> is a consulting company that specializes in data science, machine learning, and algorithm development both on the edge and in the cloud. We partner with our clients, who range from Fortune 500 companies to innovative startups, to turn their ideas into reality. Have a hard problem in mind? Get in touch at info@edgeanalytics.io.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/1*bZ7vsPoqYEPjc12QHLZWDQ.png" /></figure><h4><a href="https://infinity.ai/">Infinity AI</a>. Build better AI models faster with synthetic data.</h4><p><a href="https://infinity.ai/">Infinity AI</a> is an automated synthetic data platform for ML engineers. Their self-serve API and suite of tools make it easy for engineers to turn one real-world video into hundreds of similar (and perfectly labeled) synthetic videos for ML training. For open-source datasets and more info, visit <a href="http://infinity.ai/">infinity.ai</a>, follow on <a href="https://www.linkedin.com/company/infinityai">LinkedIn</a>, or get in touch at <a href="/service/mailto:info@toinfinity.ai">info@toinfinity.ai</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e99c25acbf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/edge-analytics/using-stable-diffusion-to-improve-image-segmentation-models-1e99c25acbf">Using Stable Diffusion to Improve Image Segmentation Models</a> was originally published in <a href="https://medium.com/edge-analytics">Edge Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting the Most Out of GPT-3-based Text Classifiers: Part Three]]></title>
            <link>https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-three-77305628f472?source=rss-6db1be83348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/77305628f472</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[gpt-3]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Alex Browne]]></dc:creator>
            <pubDate>Wed, 25 Aug 2021 12:00:28 GMT</pubDate>
            <atom:updated>2021-10-16T01:44:14.530Z</atom:updated>
            <content:encoded><![CDATA[<h4>Label Probabilities and Multi-Label Output</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T6PiBf4bFjsmJrqEfEUtnA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@louishansel?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Louis Hansel — Restaurant Photographer</a> on <a href="https://unsplash.com/s/photos/apple?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>This is part three of a series on how to get the most out of GPT-3 for text classification tasks (see <strong>part one</strong> on <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-one-797460a5556e">reducing out-of-bound predictions</a> and <strong>part two</strong> about <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-two-a869313b4206">fine-tuning prompts with probability spectrums</a>). This post will describe a technique for producing multi-label outputs and improving classification results.</p><p>At <a href="https://edgeanalytics.io/">Edge Analytics</a>, we’re using GPT-3 and other cutting-edge NLP technologies to create end-to-end solutions. Check out our <a href="https://www.invibe.co/blog/how-gpt-3-unlocks-deeper-listening-at-invibe">recent work</a> with <a href="https://www.invibe.co/">inVibe</a> using GPT-3 to improve the market research process!</p><h3>What is GPT-3?</h3><p><a href="https://arxiv.org/abs/2005.14165">GPT-3</a> stands for “Generative Pre-trained Transformer 3”. It was created by <a href="https://openai.com/">OpenAI</a> and at the time of writing is the largest model of its kind, consisting of over 175 billion parameters. It was also pre-trained using one of the largest text corpuses ever, consisting of about 499 billion tokens (approximately 2 trillion characters) which includes a significant chunk of all text available on the internet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3HLFEet1XPwEoN6D.gif" /><figcaption>In this example, the text in bold is the “prompt” and the rest of the text is GPT-3’s “prediction”.</figcaption></figure><p>GPT-3 uses a text-based interface. It accepts a sequence of text (i.e., the “prompt”) as an input and outputs a sequence of text that it predicts should come next (i.e., the “prediction” or “completion”). Through this surprisingly simple interface, GPT-3 is able to produce impressive results. The trick is designing the right prompt to extract the right knowledge encoded within GPT-3.</p><p>At the time of writing, GPT-3 is in a private beta. You have to apply for access on <a href="https://beta.openai.com/">the OpenAI website</a>. We recommend watching <a href="https://youtu.be/7_c3zjpDvqg">this YouTube video</a> for a good overview of the process and some tips for getting access.</p><h3>Interpreting Label Probabilities</h3><p>There’s one more technique that we use at <a href="https://edgeanalytics.io/">Edge Analytics</a> to get the most out of GPT-3, and it has to do with how we interpret the API output. The GPT-3 API doesn’t just return the predicted text; it also returns some metadata about the prediction, like the duration for the request and the exact version of the underlying model that was used. Of particular interest to us is a field called top_logprobs, which contains the natural log of the probability for each possible token for each part of the prediction. The format of the top_logprobs field makes it a little hard to interpret at first glance. However, with some additional processing steps we can use it to improve accuracy and convert from single-label output to multi-label output.</p><p>Let’s look at a specific example. We’ll use the following prompt:</p><pre>Classify each of the following foods as either a fruit or a vegetable.<br>Food: apple<br>Label:</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wIccTkOW1YpbZV5Y.png" /></figure><p>If we just look at the probability spectrums in the playground, we see that GPT-3’s top prediction is “fruit” with a probability of ~33%. It might not seem like GPT-3 is very confident about the predicted label. However, there’s more to it than that. As we’ll soon see, the reason that this probability is relatively low has to do with formatting. GPT-3 is not sure whether the label should be capitalized, whether it should be proceeded by a space, or whether it should be surrounded by quotation marks. In reality, “fruit”, “ fruit”, and “Fruit” have the same semantic meaning, just with minor formatting differences. We should try to account for similarities such as this in order to extract the most accurate prediction possible.</p><p>Here are the first three elements in the top_logprobs array (while this is sufficient for our example, in some cases it may be necessary to look at more than just the first three elements):</p><pre>&quot;top_logprobs&quot;: [<br>  {<br>    &quot;\n&quot;: -6.6079745,<br>    &quot; vegetable&quot;: -4.0431094,<br>    &quot; &quot;: -6.2982974,<br>    &quot; Fruit&quot;: -1.9344363,<br>    &quot; Veget&quot;: -4.888889,<br>    &quot;Ve&quot;: -10.074649,<br>    &quot;ve&quot;: -9.217822,<br>    &quot;fruit&quot;: -6.2173643,<br>    &quot;F&quot;: -8.124118,<br>    &quot; fruit&quot;: -0.1926153<br>  },<br>  {<br>    &quot;\n&quot;: -0.0059370887,<br>    &quot; vegetable&quot;: -6.9535685,<br>    &quot; &quot;: -5.4001474,<br>    &quot; Fruit&quot;: -11.35315,<br>    &quot; Veget&quot;: -8.579499,<br>    &quot;Ve&quot;: -12.559547,<br>    &quot;ve&quot;: -11.902375,<br>    &quot;fruit&quot;: -12.914117,<br>    &quot;F&quot;: -14.043984,<br>    &quot; fruit&quot;: -8.356797<br>  },<br>  {<br>    &quot;get&quot;: -6.502035,<br>    &quot;\n&quot;: -0.4551382,<br>    &quot; vegetable&quot;: -8.820861,<br>    &quot; &quot;: -4.151076,<br>    &quot;Ve&quot;: -4.243354,<br>    &quot;ve&quot;: -5.310599,<br>    &quot;fruit&quot;: -5.189734,<br>    &quot;F&quot;: -1.1317282,<br>    &quot;able&quot;: -8.674933,<br>    &quot; fruit&quot;: -7.591307<br>  }<br>]</pre><p>First, let’s convert these numbers into a format that we’re more familiar with. The documentation doesn’t exactly spell it out, but as you might have guessed the numbers in top_logprobs are simply the natural log of the probability. Converting them to the probability between 0 and 1 is straightforward:</p><pre>import numpy as np</pre><pre>def logprob_to_prob(logprob: float) -&gt; float:<br>    return np.exp(logprob)</pre><pre>&gt;&gt;&gt; logprob_to_prob(-1.9344363)<br>0.14450570373415098</pre><p>Our approach gets a little more complicated from here. We developed a recursive algorithm for iterating through each element in top_logprobs and adding up the corresponding probabilities for each label. Let&#39;s start with a function which just tells us the probability for a single label based on the given logprobs.</p><pre>def prob_for_label(label: str, logprobs: List[Dict[str, float]]) -&gt; float:<br>    &quot;&quot;&quot;<br>    Returns the predicted probability for the given label as<br>    a number between 0.0 and 1.0.<br>    &quot;&quot;&quot;<br>    # Initialize probability for this label to zero.<br>    prob = 0.0<br>    # Look at the first entry in logprobs. This represents the<br>    # probabilities for the very next token.<br>    next_logprobs = logprobs[0]<br>    for s, logprob in next_logprobs.items():<br>        # We want labels to be considered case-insensitive. In<br>        # other words:<br>        #<br>        #     prob_for_label(&quot;vegetable&quot;) =<br>        #         prob(&quot;vegetable&quot;) + prob(&quot;Vegetable&quot;)<br>        #<br>        s = s.lower()</pre><pre>        if label.lower() == s:<br>            # If the prediction matches one of the labels, add<br>            # the probability to the total probability for that<br>            # label.<br>            prob += logprob_to_prob(logprob)</pre><pre>    return prob</pre><p>Note that sometimes the tokens in logprobs are not exactly equal to the label, but rather they are <em>prefixes</em> of the label. This is where recursion comes in. For example, if the token in logprobs is “Vege”, we re-frame the problem by iterating through the rest of logprobs and searching for the rest of the label (in this case “table”).</p><pre>def prob_for_label(label: str, logprobs: List[Dict[str, float]]) -&gt; float:<br>    &quot;&quot;&quot;<br>    Returns the predicted probability for the given label as<br>    a number between 0.0 and 1.0.<br>    &quot;&quot;&quot;<br>    # Initialize probability for this label to zero.<br>    prob = 0.0<br>    # Look at the first entry in logprobs. This represents the<br>    # probabilities for the very next token.<br>    next_logprobs = logprobs[0]<br>    for s, logprob in next_logprobs.items():<br>        # We want labels to be considered case-insensitive. In<br>        # other words:<br>        #<br>        #     prob_for_label(&quot;vegetable&quot;) =<br>        #         prob(&quot;vegetable&quot;) + prob(&quot;Vegetable&quot;)<br>        #<br>        s = s.lower()</pre><pre>        if label.lower() == s:<br>            # If the prediction matches one of the labels, add<br>            # the probability to the total probability for that<br>            # label.<br>            prob += logprob_to_prob(logprob)</pre><pre>        elif label.lower().startswith(s):<br>            # If the prediction is a prefix of one of the labels, we<br>            # need to recur. Multiply the probability of the prefix<br>            # by the probability of the remaining part of the label.<br>            # In other words:<br>            #<br>            #     prob_for_label(&quot;vegetable&quot;) =<br>            #         prob(&quot;vege&quot;) * prob(&quot;table&quot;)<br>            #<br>            rest_of_label = label[len(s) :]<br>            remaining_logprobs = logprobs[1:]<br>            prob += logprob * prob_for_label(<br>                rest_of_label,<br>                remaining_logprobs,<br>            )</pre><pre>    return prob</pre><p>At this point all we need to do is call prob_for_label for each of our labels: &quot;vegetable&quot;, and &quot;fruit&quot;. What we get at the end of this process is a multi-label output, a probability between 0.0 and 1.0 for each possible label:</p><pre>{<br>  &quot;fruit&quot;: 0.78,<br>  &quot;vegetable&quot;: 0.05,<br>  &quot;other&quot;: 0.17<br>}</pre><h3>Summary</h3><p>In this post, we described a technique for using logprobs to improve the accuracy of GPT-3-based text classifiers. In the example above, this technique gives us a more complete picture of the probability for the “fruit” label. The naive approach estimated the probability to be 33%, which seemed to indicate that GPT-3 was not confident about the prediction. However, our new technique shows that the probability is closer to 78%, which tells us GPT-3 <em>is</em> fairly certain about the answer.</p><p>The second major benefit is that this technique gives us multi-label outputs, which otherwise are not provided directly by the GPT-3 API. We can use multi-label outputs to gain insight into what GPT-3 is really thinking and further refine our predictions. For example, for sentiment analysis if both “positive” and “negative” have relatively high probabilities, it might be a good indication that the most appropriate label should really be “mixed”.</p><p>Note that this example and the corresponding code have been simplified for demonstration purposes. In the real world, we have more edge cases and more complicated labels to deal with. This technique makes an even bigger difference for longer labels or labels that contain the same prefix (e.g. “somewhat positive”, “somewhat negative”, “mostly positive”, and “mostly negative”). In such cases, naive approaches to estimating label probability can sometimes produce answers that are flat out wrong, and the recursive technique we described is <em>critical</em> for ensuring accuracy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XXD8UmjZs6JzG-Hw.png" /></figure><h3>GPT-3 at Edge Analytics</h3><p><a href="https://edgeanalytics.io/">Edge Analytics</a> has helped multiple companies build solutions that leverage GPT-3. More broadly, we specialize in data science, machine learning, and algorithm development both on the edge and in the cloud. We provide end-to-end support throughout a product’s lifecycle, from quick exploratory prototypes to production-level AI/ML algorithms. We partner with our clients, who range from Fortune 500 companies to innovative startups, to turn their ideas into reality. Have a hard problem in mind? Get in touch at <a href="/service/mailto:info@edgeanalytics.io">info@edgeanalytics.io</a>.</p><p><strong>Getting the Most Out of GPT-3-based Text Classifiers:</strong> <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-one-797460a5556e">Part 1</a>, <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-two-a869313b4206">Part 2</a>, <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-three-77305628f472">Part 3</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=77305628f472" width="1" height="1" alt=""><hr><p><a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-three-77305628f472">Getting the Most Out of GPT-3-based Text Classifiers: Part Three</a> was originally published in <a href="https://medium.com/edge-analytics">Edge Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting the Most Out of GPT-3-based Text Classifiers: Part Two]]></title>
            <link>https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-two-a869313b4206?source=rss-6db1be83348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/a869313b4206</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[gpt-3]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Alex Browne]]></dc:creator>
            <pubDate>Wed, 04 Aug 2021 11:33:42 GMT</pubDate>
            <atom:updated>2021-10-16T01:43:07.781Z</atom:updated>
            <content:encoded><![CDATA[<h4>Fine-Tuning Prompts with Probability Spectrums</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lAdVkWysUaXsQ_JAxuqdQw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@markuswinkler?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Markus Winkler</a> on <a href="https://unsplash.com/s/photos/dice?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>This is part two of a series on how to get the most out of GPT-3 for text classification tasks (<a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-one-797460a5556e">Part 1</a>, <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-three-77305628f472">Part 3</a>). In this post, we’ll talk about how to use a feature of the GPT-3 playground — probability spectrums — to fine-tune prompts and improve performance.</p><p>At <a href="https://edgeanalytics.io/">Edge Analytics</a>, we’re using GPT-3 and other cutting-edge NLP technologies to create end-to-end solutions. Check out our <a href="https://www.invibe.co/blog/how-gpt-3-unlocks-deeper-listening-at-invibe">recent work</a> with <a href="https://www.invibe.co/">inVibe</a> using GPT-3 to improve the market research process!</p><h3>What is GPT-3?</h3><p><a href="https://arxiv.org/abs/2005.14165">GPT-3</a> stands for “Generative Pre-trained Transformer 3”. It was created by <a href="https://openai.com/">OpenAI</a> and at the time of writing is the largest model of its kind, consisting of over 175 billion parameters. It was also pre-trained using one of the largest text corpuses ever, consisting of about 499 billion tokens (approximately 2 trillion characters) which includes a significant chunk of all text available on the internet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jaRUMAhlHiAslOWi.gif" /><figcaption>In this example, the text in bold is the “prompt” and the rest of the text is GPT-3’s “prediction”.</figcaption></figure><p>GPT-3 uses a text-based interface. It accepts a sequence of text (i.e., the “prompt”) as an input and outputs a sequence of text that it predicts should come next (i.e., the “prediction” or “completion”). Through this surprisingly simple interface, GPT-3 is able to produce impressive results. The trick is designing the right prompt to extract the right knowledge encoded within GPT-3.</p><p>At the time of writing, GPT-3 is in a private beta. You have to apply for access on <a href="https://beta.openai.com/">the OpenAI website</a>. We recommend watching <a href="https://youtu.be/7_c3zjpDvqg">this YouTube video</a> for a good overview of the process and some tips for getting access.</p><h3>Using Probability Spectrums Effectively</h3><p>For many tasks, good prompt design is imperative for getting good results from GPT-3. Designing a good prompt can sometimes feel like a guessing game, and it’s not always clear why changing the wording of the prompt can increase or decrease performance. One technique that we can use to help improve our prompts involves looking at the full probability spectrum in the GPT-3 playground.</p><p>Let’s consider the following GPT-3 prompt. This a fairly minimal prompt for tweet sentiment classification and includes several examples.</p><pre>Label the tweets as either &quot;positive&quot;, &quot;negative&quot;, &quot;mixed&quot;, or &quot;neutral&quot;:<br>Tweet: I can say that there isn&#39;t anything I would change.<br>Label: positive<br>Tweet: I&#39;m not sure about this.<br>Label: neutral<br>Tweet: I liked some parts but I didn&#39;t like other parts.<br>Label: mixed<br>Tweet: I think the background image could have been better.<br>Label: negative<br>Tweet: I liked it.<br>Label:</pre><p>Enabling probability spectrums in the GPT-3 playground is simple: use the drop-down menu to set the “Show Probabilities” option to “Full Spectrum”. This adds color-coded highlighting to all the text in both the prompt and the predicted completion. This can be interpreted as GPT-3’s answer to the question “If the prompt stopped here, what would be the next predicted token?” for each token in the prompt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PPHhmyHWAEZUJtPFmhEfGQ.png" /><figcaption>The “Show probability” option can be found in the bottom right corner of the playground.</figcaption></figure><p>With this option enabled, the playground will display the probability for each token in the prompt by highlighting the text with different colors. Green highlighting means the token has a high probability whereas tokens that are highlighted red have a low probability. The different hues in between representing the full range of probability between 0% and 100%. We can also hover over each token to see the breakdown of token probabilities in more detail.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7KsDUUjkj1GttLcj.png" /><figcaption>GPT-3 considers the label “positive” to have a probability of 58%.</figcaption></figure><p>For the first example, hovering over the word “positive” reveals what GPT-3 would predict if the prompt were to stop here. In other words, if we didn’t tell GPT-3 the answer, what would it guess? With a probability of ~58%, it would’ve predicted “positive”, which is correct. One additional note about this first example: we can see from the probability spectrum that at this point GPT-3 isn’t sure what capitalization to use and whether or not the predicted label should be in quotes. These uncertainties will go away in subsequent examples once a pattern is established.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XOUDjjIeSBGBO8rN.png" /><figcaption>Here, GPT-3 considers the label “neutral” to be less probable at ~34%.</figcaption></figure><p>GPT-3 is less certain about the second example, where it would have predicted “negative” with a probability of ~34%. It’s not clear that GPT-3 is objectively wrong to predict “negative” in this case (it is at least arguably correct). One way to interpret the result here is that the example is providing <em>new information</em> to GPT-3. By including this example in the prompt, we are homing in on what exactly we mean by “Label the tweets as either ‘positive’, ‘negative’, ‘mixed’, or ‘unclear’”. This specific example may be showing GPT-3 that tweets that indicate uncertainty should be considered “neutral”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vkwzSI60xWhry7Fy.png" /></figure><p>For the third example, GPT-3 would have predicted “mixed” with a probability of ~49%. While this <em>is</em> the label we were looking for, GPT-3 is not very certain. The probability for “negative” is ~45% which is very close to the probability for its top choice. We can infer that this example is again providing <em>useful information</em> to GPT-3. We are showing GPT-3 that tweets which contain both positive and negative sentiment should be considered “mixed” (or at least re-affirming it).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nGM8Td7MK1n-AnJy.png" /></figure><p>In the fourth and final example, GPT-3 would have correctly predicted “negative” with a probability of ~98% — much higher than for the other examples. We can infer that this last example is not really providing any new or useful information to GPT-3, since it already knew the correct label.</p><h3>Summary</h3><p>Our standard process for using GPT-3 for new tasks often involves a process very similar to what is described above. The probability spectrums give us some hints as to which examples could potentially be removed and which examples should be elaborated on. For example, based on the results above, we might try removing the fourth example since it does not seem to be providing any new information. It might also be effective to include more examples of “neutral” and “mixed” tweets since those seem to be the categories that GPT-3 was the least certain about.</p><p>Probability spectrums are undoubtedly a useful tool for fine-tuning prompts, but it’s important to keep in mind that this technique can only offer us <em>h</em>ints<em> </em>about what to change. After making any changes, it is important to rigorously test the new prompt in order to confirm whether our inferences are correct and measure the impact on accuracy and performance. Several cycles of prompt adjustment and testing are often needed in order to yield the most optimal results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0V00C6ZPgh1ersHQk8OlrQ.png" /></figure><h3>GPT-3 at Edge Analytics</h3><p><a href="https://edgeanalytics.io/">Edge Analytics</a> has helped multiple companies build solutions that leverage GPT-3. More broadly, we specialize in data science, machine learning, and algorithm development both on the edge and in the cloud. We provide end-to-end support throughout a product’s lifecycle, from quick exploratory prototypes to production-level AI/ML algorithms. We partner with our clients, who range from Fortune 500 companies to innovative startups, to turn their ideas into reality. Have a hard problem in mind? Get in touch at <a href="/service/mailto:info@edgeanalytics.io">info@edgeanalytics.io</a>.</p><p><strong>Getting the Most Out of GPT-3-based Text Classifiers:</strong> <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-one-797460a5556e">Part 1</a>, <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-two-a869313b4206">Part 2</a>, <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-three-77305628f472">Part 3</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a869313b4206" width="1" height="1" alt=""><hr><p><a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-two-a869313b4206">Getting the Most Out of GPT-3-based Text Classifiers: Part Two</a> was originally published in <a href="https://medium.com/edge-analytics">Edge Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Best Practices]]></title>
            <link>https://medium.com/edge-analytics/python-best-practices-2934de825fd2?source=rss-6db1be83348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/2934de825fd2</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Alex Browne]]></dc:creator>
            <pubDate>Thu, 15 Jul 2021 11:02:26 GMT</pubDate>
            <atom:updated>2021-07-15T11:02:26.713Z</atom:updated>
            <content:encoded><![CDATA[<h4>The guidelines we follow at Edge Analytics for dependency management, linting, autoformatting, testing, and publishing in python</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w6D21XiaqlCAoy7Nx0AICQ.png" /></figure><p>At <a href="https://edgeanalytics.io/">Edge Analytics</a>, we specialize in data science, machine learning, and algorithm development both on the edge and in the cloud. We provide end-to-end support throughout a product’s lifecycle, from quick exploratory prototypes to production-level AI/ML algorithms. While we use a variety of tools to get the job done, we often find ourselves using Python in at least some stages of the process.</p><p>We have put together this handy guide to share our best practices for using Python. Here you will find guidelines on dependency management, linting, auto-formatting, testing, building, and publishing packages!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/0*WnifbIt-ol-Yfmo_.png" /></figure><h3>Dependency Management — Poetry</h3><p>We recommend using <a href="https://python-poetry.org/">Poetry</a> for installing dependencies and creating isolated environments between projects. There are a couple of reasons why Poetry is preferred over conda, pip and virtualenv, or other alternatives:</p><ul><li>It removes many of the pitfalls and traps found in alternative tools, which is especially important for newcomers. For example, it automatically creates a distinct virtual environment for each project and automatically adds dependencies to pyproject.toml when they are installed (no need to remember to run pip freeze).</li><li>It drastically simplifies the process for building and distributing packages (e.g. on PyPI or a private GitHub repo). Instead of managing several different config files with duplicated information, it uses a single config file and building is as simple as poetry build.</li><li>It uses a lockfile (called poetry.lock) which is critically important for guaranteeing reproducible builds and making sure things don’t break unexpectedly in production. In contrast, pip install -r requirements.txt <em>does not</em> always result in the same version of each dependency being installed.</li></ul><h4>Poetry Installation</h4><p>See <a href="https://python-poetry.org/docs/#installation">https://python-poetry.org/docs/#installation</a>.</p><h4>Poetry Basic Usage</h4><p>Note that it is not necessary to activate the virtual environment before running poetry add or poetry install. Poetry commands automatically work within the corresponding virtual environment for each project. No more accidentally using the wrong environment or forgetting to activate it 🎉</p><p>Installing required dependencies (analogous to pip install -r requirements.txt):</p><pre>poetry install</pre><p>Adding new dependencies (analogous to pip install numpy &amp;&amp; pip freeze &gt; requirements.txt):</p><pre>poetry add numpy</pre><p>Running a command from inside the virtual environment (analogous to source /.envs/myproject/bin/activate followed by python -m flake8):</p><pre>poetry run flake8</pre><p>Typically with Poetry, you don’t need to work with virtual environments directly. poetry add, poetry install, and poetry run each automatically run inside the virtual environment corresponding to the project. However, you still have the option to open a new shell inside the virtual environment if you need or want to (analogous to source /.envs/myproject/bin/activate):</p><pre>poetry shell</pre><p>Exiting the virtual environment shell (analogous to deactivate):</p><pre>exit</pre><p>See the <a href="https://python-poetry.org/docs/cli/">Poetry documentation</a> for a full list of commands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*Xxj-fbDVWOOLkHiu.png" /></figure><h3>Linting — Flake8</h3><p>We recommend using <a href="https://pypi.org/project/flake8/">Flake8</a> to lint Python code. Flake8 helps improve code quality and catches certain kinds of bugs early, before you even run any code.</p><h4>Flake8 Installation</h4><p>You can install Flake8 and some recommended plugins in a new project with:</p><pre>poetry add --dev flake8 flake8-unused-arguments</pre><p>You can run Flake8 directly from the command line:</p><pre>poetry run flake8</pre><p>However, we also strongly recommend running Flake8 automatically in your editor. There are plugins available for most major editors/IDES (see below).</p><p><strong>Flake8 Configuration</strong></p><p>Use the following configuration file, which should be called .flake8 and be located in the project root directory:</p><pre>[flake8]<br>; Minimal configuration for Flake8 to work with Black.<br>max-line-length = 88<br>extend-ignore = E203, E501</pre><p>This config file ensures compatibility with Black and includes some other sensible defaults.</p><p><strong>Flake8 Editor Integration</strong></p><ul><li><strong>Visual Studio Code: </strong>The <a href="https://code.visualstudio.com/docs/languages/python">Python plugin</a> for VS Code includes built-in support for Flake8. Simply <a href="https://code.visualstudio.com/docs/python/linting">configure it to use Flake8 as the linter</a>.</li><li><strong>Vim</strong>: Install the <a href="https://github.com/nvie/vim-flake8">vim-flake8</a> plugin. See <a href="https://github.com/nvie/vim-flake8/blob/master/README.mdown">the README</a> for additional instructions.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jENAAQ6Z7O3ZCEDU.png" /></figure><h3>Auto-formatting — Black</h3><p>We recommend using <a href="https://pypi.org/project/black/">Black</a> to automatically format Python code. Black takes care of formatting for you so that there is one less thing to worry about and argue over. If needed, you can install Black in a new project with:</p><pre>poetry add --dev black</pre><p>You can run Black directly from the command line:</p><pre>poetry run black</pre><p>However, we also strongly recommend running Black automatically in your editor. There are <a href="https://black.readthedocs.io/en/stable/editor_integration.html">plugins available for most major editors/IDES</a>.</p><p><strong>Black Editor Integration</strong></p><ul><li><strong>Visual Studio Code:</strong> The <a href="https://code.visualstudio.com/docs/languages/python">Python plugin</a> for VS Code includes built-in support for Black. Simply <a href="https://code.visualstudio.com/docs/python/editing#_formatting">configure it to use Black as the formatter</a>.</li><li><strong>Vim:</strong> Follow the corresponding <a href="https://black.readthedocs.io/en/stable/integrations/editors.html#vim">instructions from the Black docs</a>.</li><li><strong>Other editors: </strong>Refer to <a href="https://black.readthedocs.io/en/stable/integrations/editors.html">the Editor Integration page</a> from the Black docs.</li></ul><h3>Testing — PyTest</h3><p>Testing is a critically important part of writing maintainable production-level code. Any code which is going to stick around and be used by others (either internally or externally) should be tested.</p><h4>Types of Tests</h4><p>There are many types of tests in the software development world, but the two main categories are “unit tests” and “integration tests”. <a href="https://www.testim.io/blog/unit-test-vs-integration-test/">This blog post</a> goes into depth about the difference between the two. To quickly summarize:</p><ul><li>Unit tests operate on the smallest individual units of code: functions. They are used to ensure that a function does what it is supposed to do when given different inputs. Typically, unit testing should focus on functions without any side-effects.</li><li>Integration tests operate at a higher level and test integrations <em>between</em> different functions or between different APIs and services (e.g. a database, mailserver, or third-party API). They can sometimes have similar entrypoints (e.g. calling a function), but may also require a different, more complicated testing process (e.g. spinning up a full environment with a local server and database).</li></ul><h4>Installing and Using Pytest</h4><p>We recommend using <a href="https://docs.pytest.org/en/stable/index.html">pytest</a> for testing in Python. If needed, you can install pytest in a new project with:</p><pre>poetry add --dev pytest</pre><p>You can run pytest directly from the command line:</p><pre>poetry run pytest</pre><p>See <a href="https://docs.pytest.org/en/stable/index.html">the official docs</a> for more information about how to use pytest.</p><h3>Building and Publishing</h3><p>If you are using poetry for dependency management, building and publishing a package is dead simple and does not require any additional configuration files.</p><h4>Versioning</h4><p>Any code which is published as a package or deployed as a server should follow <a href="https://semver.org/">semantic versioning</a>.</p><p>The poetry version command automatically bumps the version number for a patch, minor, or major release. For example, when building and publishing a patch release, you should run:</p><pre>poetry version patch</pre><p>This will automatically bump the version in pyproject.toml.</p><h4>Building</h4><p>After bumping the version, simply run:</p><pre>poetry build</pre><p>This will add both a source code distribution and a binary distribution (wheel).</p><h4>Publishing</h4><p>While not all code needs to be published, publishing code as a Python package is the best way to share and re-use it across different projects or even different companies/organizations. There are two different ways of publishing packages: public and private.</p><ul><li>Public packages are for open source code that is shared with everyone.</li><li>Private packages are shared only with a select group of people (i.e. people from your company or organization).</li></ul><h4>Public Packages</h4><p>To publish a package to the public PyPI index, you first need to <a href="https://pypi.org/account/register/">register for a PyPI account</a>, then <a href="https://pypi.org/help/#apitoken">set up API tokens</a>.</p><p>To publish, simply run:</p><pre>poetry publish</pre><p>You can then install it the same way as any other public package:</p><pre>poetry add my-package-name</pre><h4>Private Packages</h4><p>For packages that you don’t want to be public, you can also publish to a private GitHub repository. To do that, simply push the changes to GitHub after building. We also recommend assigning a tag in GitHub corresponding to the version number, for example:</p><pre>git tag v1.0.2<br>git push --tags</pre><p>After pushing to the GitHub repository, the package can be installed with:</p><pre>poetry add git+ssh://git@github.com/your-username-or-organization/your-package.git#main</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vGFsxN7-JLoxVKGD.png" /></figure><p><a href="https://edgeanalytics.io/">Edge Analytics</a> is a company that specializes in data science, machine learning, and algorithm development both on the edge and in the cloud. We provide end-to-end support throughout a product’s lifecycle, from quick exploratory prototypes to production-level AI/ML algorithms. We partner with our clients, who range from Fortune 500 companies to innovative startups, to turn their ideas into reality. Have a hard problem in mind? Get in touch at <a href="/service/mailto:info@edgeanalytics.io">info@edgeanalytics.io</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2934de825fd2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/edge-analytics/python-best-practices-2934de825fd2">Python Best Practices</a> was originally published in <a href="https://medium.com/edge-analytics">Edge Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting the Most Out of GPT-3-based Text Classifiers: Part One]]></title>
            <link>https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-one-797460a5556e?source=rss-6db1be83348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/797460a5556e</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[gpt-3]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Alex Browne]]></dc:creator>
            <pubDate>Thu, 01 Jul 2021 12:02:22 GMT</pubDate>
            <atom:updated>2021-10-16T01:42:15.293Z</atom:updated>
            <content:encoded><![CDATA[<h4>Reducing Out-of-Bounds Predictions</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*_Yo4QbDJAlnEYJWKhiKacw.jpeg" /><figcaption>Source: <a href="https://thehustle.co/07202020-gpt-3/">https://thehustle.co/07202020-gpt-3/</a></figcaption></figure><p>This is part one of a series on how to get the most out of GPT-3 for text classification tasks (<a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-two-a869313b4206">Part 2</a>, <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-three-77305628f472">Part 3</a>). In this post, we’ll talk about a common issue, “out-of-bounds” predictions, and how to reduce or even completely eliminate it.</p><p>At <a href="https://edgeanalytics.io/">Edge Analytics</a>, we’re using GPT-3 and other cutting-edge NLP technologies to create end-to-end solutions. Check out our <a href="https://www.invibe.co/blog/how-gpt-3-unlocks-deeper-listening-at-invibe">recent work</a> with <a href="https://www.invibe.co/">inVibe</a> using GPT-3 to improve the market research process!</p><h3>What is GPT-3?</h3><p><a href="https://arxiv.org/abs/2005.14165">GPT-3</a> stands for “Generative Pre-trained Transformer 3”. It was created by <a href="https://openai.com/">OpenAI</a> and at the time of writing is the largest model of its kind, consisting of over 175 billion parameters. It was also pre-trained using one of the largest text corpuses ever, consisting of about 499 billion tokens (approximately 2 trillion characters) which includes a significant chunk of all text available on the internet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MJQnTqewQdR-Eyd4uiSkHQ.gif" /><figcaption>In this example, the text in bold is the “prompt” and the rest of the text is GPT-3’s “prediction”.</figcaption></figure><p>GPT-3 uses a text-based interface. It accepts a sequence of text (i.e., the “prompt”) as an input and outputs a sequence of text that it predicts should come next (i.e., the “prediction” or “completion”). Through this surprisingly simple interface, GPT-3 is able to produce impressive results. The trick is designing the right prompt to extract the right knowledge encoded within GPT-3.</p><p>At the time of writing, GPT-3 is in a private beta. You have to apply for access on <a href="https://beta.openai.com/">the OpenAI website</a>. We recommend watching <a href="https://youtu.be/7_c3zjpDvqg">this YouTube video</a> for a good overview of the process and some tips for getting access.</p><h3>What is an out-of-bounds prediction?</h3><p>This post describes some techniques we use when leveraging GPT-3 for text classification tasks. One common issue is that GPT-3 can produce outputs that are not one of the intended classes. For example, when we designed a GPT-3-based sentiment classifier to label text as either “positive”, “mixed”, or “negative”, it would it would sometimes predict “unknown”. This label makes sense semantically but is not one of the three labels we expected. We call this kind of output from GPT-3 an “out-of-bounds” prediction.</p><p>The reason “out-of-bounds” predictions can occur has to do with how GPT-3 was trained and what it is designed to do. GPT-3 is not <em>just</em> a text classifier; it doesn’t even have any built-in rules for it! GPT-3 was really only designed to do one thing: predict the sequence of text that is most likely to follow the prompt. It has to learn the rules of the classifier on the fly based on the prompt and any similarities or patterns from its massive corpus. Given this context, it’s understandable that it might get things wrong sometimes.</p><p>Luckily, the GPT-3 API gives us some knobs to tweak to <em>almost</em> entirely eliminate out-of-bounds predictions. In this blog post, we’ll cover two of them:</p><ol><li>Adjusting the temperature.</li><li>Setting the logit_bias.</li></ol><h3>Adjusting the temperature</h3><p>The temperature parameter is pretty straightforward. <a href="https://beta.openai.com/docs/api-reference/completions/create#completions/create-temperature">The GPT-3 documentation</a> explains how temperature works: &quot;Higher values means the model will take more risks. Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer.&quot; For text classification tasks, you usually want to set this to 0.</p><p>While setting temperature to 0 goes a long way to reduce out-of-bounds predictions, it doesn&#39;t completely eliminate them. To take things one step further, you can use the logit_bias parameter (more on that in a bit).</p><h3>The logit_bias parameter</h3><h4>Tokenization: How GPT-3 sees text</h4><p>In order to understand how logit_bias can be used to create a whitelist of labels, we first need to take a step back and explain how GPT-3 interprets text. GPT-3 doesn&#39;t see text as a string of characters or words, but as a list of &quot;token ids&quot;. Each token id is a unique number which represents a short sequence of characters (on average each token corresponds to about 4 characters of English text). The process for converting text to tokens is based on the frequency of certain combinations of characters. Common words like &quot;cat&quot; and &quot;ball&quot; are likely to be assigned their own unique token, whereas less common words like &quot;establishment&quot; are likely to be sliced up into more than one token (e.g., &quot;estab&quot;, &quot;lish&quot;, &quot;ment&quot;).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5SFSuFOM8GMKqIIDcz5sZQ.png" /><figcaption>GPT-3 breaks sees text as a series of tokens, each with a unique token_id.</figcaption></figure><p>As to why GPT-3 uses this tokenization approach? Well, it has a few benefits. It provides a fairly dense encoding for text that computers can easily understand. Since every word can be constructed by a combination of tokens, no words are considered completely out-of-vocabulary (a problem that can sometimes plague other encoding techniques like word embeddings). GPT-3 can accept as input words that it has never seen before, and given enough context, can potentially understand them.</p><h4>Setting the logit_bias parameter</h4><p>The logit_bias parameter is a bit more complicated than temperature. Effectively, it can be used to tweak the probability of certain tokens being included in GPT-3’s prediction. As <a href="https://beta.openai.com/docs/api-reference/completions/create#completions/create-temperature">the documentation elaborates</a>, “values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token”. For most text classification tasks, we want to restrict GPT-3’s output to a whitelist of labels that we define ahead of time. In other words, “exclusive selection” is exactly what we want.</p><p>So in order to use logit_bias to create a whitelist of labels, we need to first convert our labels to tokens. For this, we’ll need to import the <a href="https://pypi.org/project/transformers/">transformers</a> library.</p><pre>from transformers import GPT2TokenizerFast<br>tokenizer = GPT2TokenizerFast.from_pretrained(&quot;gpt2&quot;)</pre><pre>def tokenize_labels(labels: List[str]) -&gt; List[str]:<br>    &quot;&quot;&quot;<br>    Converts a list of labels into a list of GPT-3 tokens.<br>    &quot;&quot;&quot;</pre><pre>    tokens = []<br>    for label in labels:<br>        tokens += tokenizer.encode(label)</pre><pre>    return tokens</pre><p>But there’s a catch! Because of how token-encoding works, “ positive” (with a preceding space) is represented by a different token than “positive” (without the space). To make matters more complicated, it’s not always easy to tell which representation GPT-3 will choose, since it can depend on the preceding text. We’ve found the most reliable way to account for this difference is to add <em>both</em> “positive” and “ positive” to our whitelist. We do this for each label.</p><pre>def tokenize_labels(labels: List[str]) -&gt; List[str]:<br>    &quot;&quot;&quot;<br>    Converts a list of labels into a list of GPT-3 tokens.<br>    Adds preceding whitespace as needed in order to account for<br>    quirks in how GPT-3 handles tokenization.<br>    &quot;&quot;&quot;</pre><pre>    # Start with whitespace tokens<br>    tokens = [&quot; &quot;, &quot;\n&quot;]</pre><pre>    # Tokenize each label by itself *and* with a preceding space.<br>    for label in labels:<br>        tokens += tokenizer.encode(label)<br>        tokens += tokenizer.encode(&quot; &quot; + label)<br><br>    return tokens</pre><p>The final step is to convert this list of tokens into the dictionary format expected for the logit_bias parameter, where each token has a weight (or bias) of 100. Effectively we are telling GPT-3 to exclusively construct its prediction from this set of tokens.</p><pre>def get_logit_bias(labels: List[str]) -&gt; Dict[str, float]:<br>    &quot;&quot;&quot;<br>    Returns a logit_bias that can be used to constrain GPT-3<br>    predictions to a set of pre-determined character sequences<br>    (i.e. phrases or words). Intended to be used for classification<br>    problems.<br>    &quot;&quot;&quot;</pre><pre>    tokens = tokenize_labels(labels)<br>    logit_bias: Dict[str, float] = {}<br>    for token in set(tokens):<br>        # Set the logit_bias for each token to 100, effectively<br>        # forcing GPT-3 to only choose from these tokens.<br>        logit_bias[str(token)] = 100</pre><pre>    return logit_bias</pre><p>The reason this technique does not <em>completely</em> eliminate out-of-bounds responses is that, hypothetically, there is nothing stopping GPT-3 from combining the tokens in our whitelist in unexpected ways. For example, it might combine some of the tokens for “negative” and “mixed” to output “negamixed”. In practice it is unlikely that GPT-3 will predict a high probability for made up words, and these kinds of out-of-bounds predictions are exceedingly rare (though it does depend on the exact labels used).</p><h3>Summary</h3><p>In this post, we covered two ways to reduce out-of-bounds predictions when using GPT-3 as a text classifier. First, set the temperature parameter to 0. Second — and this is a bit more complicated — use the logit_bias parameter to force GPT-3 to exclusively build its prediction from a predefined set of tokens.</p><p>To put everything together, let’s look at a specific demonstration of how these techniques can be used to improve the performance of a GPT-3-based text classifier. In this simplified example, we want GPT-3 to classify foods as either a “fruit” or a “vegetable”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7oSvffShy5K9SojUaMZzLg.gif" /><figcaption>Without guidance, GPT-3 doesn’t know what we are asking for and its predictions are somewhat random.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i6HjmUmkofYDkNScPuyBmA.gif" /><figcaption>By using the techniques described in this blog post, we can get the predictions we’re looking for.</figcaption></figure><h3>GPT-3 at Edge Analytics</h3><p><a href="https://edgeanalytics.io/">Edge Analytics</a> has helped multiple companies build solutions that leverage GPT-3. More broadly, we specialize in data science, machine learning, and algorithm development both on the edge and in the cloud. We provide end-to-end support throughout a product’s lifecycle, from quick exploratory prototypes to production-level AI/ML algorithms. We partner with our clients, who range from Fortune 500 companies to innovative startups, to turn their ideas into reality. Have a hard problem in mind? Get in touch at <a href="/service/mailto:info@edgeanalytics.io">info@edgeanalytics.io</a>.</p><p><strong>Getting the Most Out of GPT-3-based Text Classifiers:</strong> <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-one-797460a5556e">Part 1</a>, <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-two-a869313b4206">Part 2</a>, <a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-three-77305628f472">Part 3</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=797460a5556e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/edge-analytics/getting-the-most-out-of-gpt-3-based-text-classifiers-part-one-797460a5556e">Getting the Most Out of GPT-3-based Text Classifiers: Part One</a> was originally published in <a href="https://medium.com/edge-analytics">Edge Analytics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[0x Roadmap 2019 (part 3): Networked Liquidity & 0x Mesh]]></title>
            <link>https://medium.com/@albrow/0x-roadmap-2019-part-3-networked-liquidity-0x-mesh-9a24026202b3?source=rss-6db1be83348f------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a24026202b3</guid>
            <category><![CDATA[cryptocurrency]]></category>
            <category><![CDATA[ethereum]]></category>
            <category><![CDATA[0xroadmap]]></category>
            <category><![CDATA[0xproject]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Alex Browne]]></dc:creator>
            <pubDate>Wed, 27 Mar 2019 18:54:26 GMT</pubDate>
            <atom:updated>2019-03-27T19:33:42.698Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Introducing 0x Mesh, a peer-to-peer communication network for the 0x ecosystem</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rT4BX6m5romwUOV6bIG0BQ.png" /><figcaption><em>This is the third post in a series that discusses the different areas of focus for the 0x Core Team in 2019. Read part </em><a href="https://blog.0xproject.com/0x-roadmap-part-1-zeips-d5cb8458584e"><em>1</em></a><em> and </em><a href="https://blog.0xproject.com/0x-roadmap-2019-part-2-scalability-r-d-c0fc2d5101e5"><em>2</em></a><em> to learn how 0x Mesh fits into the greater 0x stack.</em></figcaption></figure><p>Today, we’re excited to introduce 0x Mesh, a peer-to-peer network for sharing orders which will serve as an alternative to the Standard Relayer API. 0x Mesh will massively reduce the effort required to tap into the 0x networked liquidity pool and the technical work associated with maintaining a compliant endpoint. Our plans to explore the use of a distributed peer-to-peer network for propagating orders go all the way back to the original <a href="https://0x.org/pdfs/0x_white_paper.pdf">0x white paper</a>. Since then, we’ve picked up a great deal of knowledge around how such a network should be designed. We are excited to share what we’ve learned and what we are building.</p><p>One of the most compelling aspects of 0x protocol is its ability to facilitate <a href="https://blog.0xproject.com/front-running-griefing-and-the-perils-of-virtual-settlement-part-2-921b00109e21">networked liquidity</a>: the seamless flow of orders through a network of interconnected marketplaces and dApps. The Radar team has <a href="https://wallet2wallet.io/">described</a> networked liquidity as a paradigm shift that transforms the term “exchange” from a noun to a verb. We agree with this assessment. <br> <br>While market participants tend to meet in specific locations to facilitate more efficient markets, networked liquidity allows orders to flow outward through a growing number of web3 capillaries, reaching a broader cross-section of users. These interconnections act as a source of <a href="https://a16z.com/2016/03/07/all-about-network-effects/">network effects</a> that lead to a greater number of opportunities for market makers and better prices for end-users. <br> <br>Over the past 18 months, the 0x Core Team has shipped key components needed to facilitate networked liquidity. <a href="https://0x.org/instant">0x Instant</a> allows orders to flow to a variety of surfaces including popular Ethereum wallets (MyCrypto), dApps (Augur), and blockchain explorers (Coingecko, EtherScan). <a href="https://0x.org/launch-kit">0x Launch Kit</a> provides a simple way for developers to launch a custom digital asset marketplace in a matter of minutes. The one underlying component that all of these products have heavily relied upon is the <a href="https://blog.0xproject.com/introducing-the-0x-standard-relayer-api-8a37bd90a3e">Standard Relayer API (SRA)</a>, which provides guidelines for sharing orders between endpoints. Moving forward, 0x Mesh can connect these products and tools in a more seamless way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QQYuPdxUjfMSjQ774fZcSw.png" /></figure><h3>The Standard Relayer API (SRA) and its challenges</h3><p>The SRA is an API specification that is meant to be implemented by relayers as a way to easily share liquidity with others. It allows 0x Instant integrations and market makers to use a single client to pull liquidity from multiple relayers. In order to make this possible, every relayer must implement the same set of HTTP/WebSocket endpoints according to the SRA specification. However, since most relayers use different programming languages and server frameworks, this implementation work is largely duplicative. Whenever a new version of the SRA is released, every relayer must update their own implementation to satisfy the new interface. Additionally, plugging into every additional SRA endpoint is a manual process that each participant must do, making it harder to join the networked liquidity pool. As evidenced by the above description, this process requires a huge amount of coordination and effort. One solution to the coordination problem is for the relayers to connect to and rely on a centralized aggregator, but this eliminates the benefits of decentralization.<br> <br>After conducting a study on the compliance of various SRA implementations, we found that most were inconsistent or only partially compliant with the API specification. This greatly reduces the utility of the SRA since developers cannot build clients and applications that rely on the specified behaviors described in the API specification. It turns out that it is non-trivial to implement the API to spec, and it would be preferable to have everyone use a single implementation.<br> <br>When we released the first version of the SRA, it was built with fungible (ERC-20) tokens in mind. With the launch of V2, 0x has become a multi-asset protocol and the need to support multiple token types has started to test the limits of the SRA. Fitting NFTs (ERC-721) into the SRA design has already proven challenging and no one knows what the next 20 token standards added to 0x will look like, or what the best interface will be for querying them. Decoupling order sharing from order querying could alleviate these issues.</p><h3>The solution: A P2P network for sharing orders</h3><p>We plan to offer an alternative to the SRA in the form of a 0x Mesh node that can be run by anyone wanting to share orders. Because every node shares the same codebase, they are able to communicate without any API compliance issues, inconsistencies, or requirements to write redundant code. Additionally, the node will automatically take care of finding and connecting to peers interested in sharing orders for a particular market, removing the need for explicit coordination by participants. Lastly, the node operator can subscribe to a stream of order updates, emitting an event whenever a new order is discovered or an existing order’s status changes (filled, canceled, expired, etc…). Node operators can decide how to transform and ingest this stream of order data from their 0x Mesh node, optimizing how it can be queried for their particular use-case. This level of querying flexibility will make it easier for the 0x protocol to seamlessly incorporate new functionality, such as new token standards and AssetProxies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZTsZ00xwPmH7YS6TVxxF3A.png" /></figure><h3>How this benefits relayers</h3><p><strong>Effortlessly surface orders across the web</strong><br> <br>The largest value proposition to relayers is that 0x Mesh will help surface their orders throughout the Internet. Unlike the SRA, 0x Mesh features a push-based messaging system and a dynamic set of peers which do not need to be known ahead of time. This makes it much easier to connect to and share orders with other members of the 0x ecosystem.</p><p>This is particularly relevant for 0x Instant, where currently hosts need to manually plug in an SRA endpoint for each relayer they want to connect to. As the number of projects that utilize 0x Instant increase, it will eventually become infeasible for each host to keep track of all the SRA endpoints they want to include. With 0x Mesh, integrators can connect to a neutral platform that aggregates orders across many relayers. In the near-term, this will likely involve an “SRA Bridge” which provides an SRA endpoint on top of 0x Mesh. This means relayers won’t need to manually contact each integrator in order to reach their audiences; it will happen automatically.<br> <br><strong>Implementing &amp; maintaining SRA becomes optional</strong><br> <br>Currently, the only way to share orders with and receive orders from other relayers is to implement the SRA. 0x Mesh will provide an additional way to contribute that does not involve implementing and maintaining a standard interface. <strong>Simply spin up a Docker container, and you can already submit orders to the network and listen for new orders.</strong> Relayers which use 0x Mesh will no longer need to worry about listening for order-related events on the Ethereum blockchain. Under the hood, each 0x Mesh node will use OrderWatcher to validate orders and remove any invalid ones.<br> <br><strong>Become a valuable access point into the network</strong><br> <br>A third, more subtle benefit stems from how the 0x Mesh will be implemented. Each node in the network will score its neighbors based on the quantity and quality (validity, novelty, etc.) of the orders it shares. Since relayers provide a single-access point into the network for their users, they will naturally move towards the core of the network graph, and as a result, will be able to offer a better trading experience. More concretely, their users will find out about orders before others, and will also have their orders more rapidly discovered than those of peripheral nodes. This advantage is especially pronounced in highly liquid markets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NYE7pc6iV_xLM7Ye87nplA.png" /></figure><h3>Towards a Unified network transport layer</h3><p>0x protocol consists of an order schema — which describes how order information is arranged into a packet — and a canonical system of Ethereum contracts which consume and process these orders. However, there are two important aspects that don’t fall into the scope of 0x protocol: (1) the network transport layer, i.e., where orders are communicated and (2) the message passing semantics, which describe how market participants coordinate trades (i.e. relayer models). While we still believe that 0x protocol should be agnostic to the network transport layer, 0x Mesh could greatly reduce friction and allow liquidity to find its way to different surfaces. Eventually, we plan to extend 0x Mesh to support any arbitrary DeFi protocol which uses off-chain messages.</p><h3>A Browser-friendly node</h3><p>We have already mentioned that 0x Mesh can be run as a Docker container, and we expect that this is how most people will interact with the network, especially in the near-term. In this model, relayers run their own backend services for connecting to 0x Mesh, maintaining a consistent view of the order book, and ultimately serving a UI to their users. The traditional relayer model comes with a lot of benefits and we believe it will continue to be the best solution for highly-liquid and well-established markets.</p><p>Instead of a Docker container, 0x Mesh can also run directly in a browser. This introduces the possibility of a new type of “serverless relayer” which only requires a front-end UI and doesn’t require any backend services. In this model, each user independently maintains their own order book and shares orders directly with other peers in the 0x Mesh network. The code could even be stored in a decentralized way (e.g. on IPFS) and accessed via a dApp browser. This will allow for frictionless market formation for the long-tail of assets that are not or cannot be serviced by relayers.</p><p>As exciting as this possibility is, the serverless relayer model comes with some downsides and is not appropriate for all markets. The first time users interact with a serverless relayer, they will experience a “warm-up delay” while the underlying 0x Mesh node finds peers to connect to and gathers orders. In addition, highly liquid markets could place a significant strain on a user’s internet bandwidth and storage space. Such markets also increase the likelihood of accidental order collisions (when two people try to fill an order at the same time). For these reasons, there is absolutely still a place in the ecosystem for traditional relayers. We expect that once a market matures to a certain level of liquidity, traditional relayers will step in to provide a more reliable experience and expose the market to a larger demographic of users.</p><h3>Technical Details and Timeline</h3><p>We are planning to launch a beta version of 0x Mesh with a handful of early participants near the end of Q2 2019. The 0x Mesh beta will be written in Go and will be compiled to WebAssembly when run in the browser. It will use WebRTC as a first-class protocol for inter-node communication (both browser and non-browser based nodes) and will rely on signaling servers to bootstrap connections between peers. We have developed a novel order eviction algorithm that precludes the need for explicit staking while preventing malicious nodes from filling up the available space on other nodes and evicting the orders of benign users. We will be releasing a comprehensive technical specification with descriptions and justifications for all our design choices in a <a href="https://forum.0x.org/">forum</a> post soon, so stay tuned!</p><h4>Learn more about 0x and join our community</h4><p><a href="https://0x.org/">Website</a> | <a href="https://blog.0xproject.com/">Blog</a> | <a href="https://twitter.com/0xproject">Twitter</a> | <a href="https://discord.gg/d3FTX3M">Discord Chat</a> | <a href="https://www.facebook.com/0xProject">Facebook</a> | <a href="https://www.reddit.com/r/0xproject">Reddit</a> | <a href="https://www.linkedin.com/company/0x">LinkedIn</a> | <a href="https://0xprotocol.substack.com/welcome">Subscribe</a> to our newsletter for updates in the 0x ecosystem</p><h4>We are hiring!</h4><p>If you’re interested in working with the 0x Core Team, check out our open positions on our<a href="https://0x.org/about/jobs"> job board</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a24026202b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>