<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Roman Kucera on Medium]]></title>
        <description><![CDATA[Stories by Roman Kucera on Medium]]></description>
        <link>https://medium.com/@roman.kucera?source=rss-68a794fa3f11------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*1KvKIkqZCN1Cgp0BJVdaCw.jpeg</url>
            <title>Stories by Roman Kucera on Medium</title>
            <link>https://medium.com/@roman.kucera?source=rss-68a794fa3f11------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 14:21:59 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@roman.kucera/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Lessons learned from organising an internal OpenAI Hackathon]]></title>
            <link>https://medium.com/@roman.kucera/lessons-learned-from-organising-an-internal-openai-hackathon-23c6846d5127?source=rss-68a794fa3f11------2</link>
            <guid isPermaLink="false">https://medium.com/p/23c6846d5127</guid>
            <category><![CDATA[hackathons]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Roman Kucera]]></dc:creator>
            <pubDate>Fri, 25 Aug 2023 16:23:51 GMT</pubDate>
            <atom:updated>2023-08-25T16:23:51.700Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hq4giABuTbS6RwH4xSWWpA.png" /></figure><p>For me, the last few months have been about generative AI and, possibly, for many of you as well. After working on adopting large language models at Ataccama, I decided to share a little about our journey. This will be a series of posts — covering different steps along the way.</p><p>Other posts in the series so far:</p><ul><li><a href="https://medium.com/@roman.kucera/what-is-gpt-and-why-it-became-so-popular-this-year-4bf3d68e3c7a">What is GPT and why it became so popular this year?</a></li><li><a href="https://medium.com/@roman.kucera/can-gpt-help-with-data-management-3f65fdf5e62c">Can GPT help with data management?</a></li></ul><p>As I wrote in the last article, earlier this year we started exploring how the current Large Language Model (LLM) hype is relevant in the space of Data Management and we were very positively surprised with how simple it was to show that there is some value there. In April, we started a serious discussion on how best to leverage the opportunity that LLMs bring to the table.</p><p>One of the ideas I started pushing at first was organising an internal hackathon, encouraging everyone to try it out, to quickly verify multiple ideas, to get people excited. At first some people tried to discourage me, questioning whether it is the best investment of our effort. Typical arguments included</p><ul><li><strong>Hackathons are a lot of work</strong> — this is true of course, as with many other things that are still worth doing.</li><li><strong>Summer is here, there is not enough time to plan it properly and people will not come</strong> — this was partially true, we ended up with around forty something people participating, which is a little less than I hoped for, but still decent enough. But there is always something, and waiting for September with the speed these technologies are developing just seemed like a wrong decision.</li><li><strong>Shouldnt we rather focus on building a dedicated AI team that would deliver a production grade feature with LLMs, rather than just playing around?</strong> — Well, my answer to this one is a bit more complicated. I think LLMs might be bringing such a big and fundamental change to the world that I do not want to work with them in a silo. I think we need everyone excited, everyone aware of what these technologies mean, what is possible today. And while some engineers are naturally excited about new technologies and actively monitoring all the trends, it is definitely not the case for everyone. And just talking to people, I realised many either do not believe LLMs are a big deal, or they think something like “this is cool, lets see what somebody else can do with it, I am too busy to care right now”. And I wanted to change this ratio a bit, I wanted as many people excited and ready to embrace this new world. Of course we need a team to build the first production ready LLM driven feature as well. But I also want everyone think where LLMs can be used to make us more effective and to deliver better value to our customers.</li></ul><p>Long story short, I pushed through and decided to organise the Hackathon (with a lot of help of course, not to steal anyones credit).</p><blockquote><strong>TIP 1:</strong> If you are a big hackathon believer like myself, just take the initiative… at first, I expected I would just suggest the Hackathon idea and it would magically happen. I actually thought that I might participate in one of the teams. But it takes a hackathon believer to organise one.</blockquote><h3>Phase 1: Planning the Hackathon</h3><p>What does it take to plan a hackathon? I’ve done my share of hackathons in the past, including helping organise a couple of them, but I would hardly consider myself an expert. That being said, I trust my gut a bit more than your average “Top 10 tips for running a hackathon” article you can find online. Let me share a few decisions we had to make.</p><h3>Main goals of the hackathon</h3><p>The first thing was to agree and the actual purpose of the hackathon, what are we trying to achieve, as this can really steer the event in a slightly different direction.</p><p>For us this came to a couple of simple points:</p><ul><li>To allow as many people as possible to experiment with the technology and inspire them to work with AI.</li><li>To demonstrate that AI is a reality.</li><li>To initiate a cross-departmental innovation process.</li><li>To identify some easy wins that can be implemented within the next couple of months.</li></ul><h3>Who is the target audience?</h3><p>This was rather simple considering the goals we put there — it was clear from day one that we want potentially anyone within the organisation to participate, any department, any office.</p><p>Even though we took this decision as a no-brainer, it proved very difficult to achieve. Seems people believe hackathons are mostly for engineers and other super-technical folks. But we believed LLMs affect everyone and that we could improve things in sales, or finance, or legal. We needed to get these people excited as well.</p><blockquote><strong>TIP 2: </strong>Keep explaining, ideally on clear examples, how people can participate. Talk to them repeatedly, don’t just assume they will sign up. I had many people who found the idea of the hackathon interesting but were too shy to sign up. Or some even signed up and got cold feet later and excused themselves last minute for reasons like “I do not know python, I dont think I can help with anything.” I managed to to convince some of them that doing something with LLMs is not just about coding, its also about expertise, framing the problem correctly, and then explaining it to the language model in plain english. You need a well-balanced team. And I think none of the people I convinced regretted the decision in the end.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bjUxYMWk4tQ-GnO9ITWzFg.png" /><figcaption>The purpose and target audience as presented to the whole company.</figcaption></figure><h3>Where to do the hackathon?</h3><p>This part depends on your budget and on where your people are located. We decided on a hybrid event, opening some of our offices for the teams that could meet in person. I believe that in-person hackathons are a much better experience in general, but we are a global company and flying over people from all over the world just for the hackathon did not seem reasonable. Doing the hackathon in one office was a possibility but we wanted the hackathon to connect people together, not to make the geographical moats even bigger. That being said, most people formed local teams and met in person at least for part of the event.</p><h3>Timing</h3><p>We started seriously planning the Hackathon at the beginning of May. We wanted to run it before summer holiday and that didn&#39;t give us much space to be honest. Since we wanted people from across the company, it wasn&#39;t easy to time it well, with all the other events already scheduled. In the end, we decided to run the hackathon from Thursday afternoon (in our main timezone), but to have the wrap-up on Monday. This allowed for some flexibility — we expected some teams would work the full time, some would only work Thursday evening + Friday, while some might work only over the weekend if their current assignment did not allow them to take Friday “off”.</p><p>In the end, we ran the hackathon from June 8th, meaning we had about 5 weeks between the first internal hackathon planning meeting and the event itself. People knew about the final date for less than a month, which in retrospect wasn’t enough — many already had their weekend plans.</p><blockquote><strong>TIP 3:</strong> If you want bigger crowd, share the date at least a month before the event, ideally more … this may sound like a no-brainer, but I think its fairly common, that these events are planned rather late.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n4AECWmkrvvxpR3h40yHhg.png" /><figcaption>The Hackathon Timeline explained</figcaption></figure><h3>Collecting Ideas</h3><p>We started by announcing the Hackathon and creating a slack channel where people should post ideas on how we could leverage generative AI in Ataccama. We promoted this heavily, including some prizes for some of the participants (based on votes + also a lottery). In the end, we collected about 50 distinct ideas. These were not necessarily ideas that would later be implemented in the hackathon, but it did work very well to start the conversation.</p><blockquote><strong>TIP 4: </strong>Collecting ideas in a slack channel sounds like a very simple thing to do. But if you get enough ideas it becomes important to curate them as well — you will get a lot of duplicates, or some ideas being superset of another idea, or some ideas described in a very ambiguous way. Some ideas will even be built around wrong assumptions, or maybe on topics that we have already solved — which is of course OK, we want to encourage everyone to post their ideas and not to be afraid to be wrong. I took the effort together with one colleague to curate the list of ideas, initiate the discussion wherever needed and together with original authors rewrite all ideas in a consistent manner. This meant linking them to similar ideas, noting some dependencies or assumptions, etc. This was much more work than I originally expected, but I think it made the final list much easier to digest.</blockquote><h3>Resources to use for the hackathon — Tooling and Data</h3><p>Our AI community did some preliminary research on what kind of tooling could be useful. We shared a fairly comprehensive list that included all sorts of models, frameworks and utilities, but in the end we decided to prepare an environment with preselected tooling configured, and with some training materials available, so that everyone can just start playing with the language models and not worry too much about everything else.</p><p>This setup included:</p><ul><li>A shared Azure OpenAI Services resource with all the available models deployed (including GPT-4, which wasnt all that common back then).</li><li>Prepared jupyter notebook with a simple project that used LangChain, Instructor embeddings and Chroma database to answer questions based on some pdf documents.</li><li>A collection of datasets that we thought might be interesting for the hackathon, in a format that would be easy to process, so that people can just start working with it without worrying too much about how to get the data. We also had the content pre-approved for usage with our security team, so that people did not have to worry whether they can actually use that data with a public OpenAI service. This data included mainly a subset of our Notion knowledgebase, our public documentation, some of our Jira projects, and an export of selected Slack channels.</li></ul><p>These things together made it super easy to start playing with LLMs even for non-engineers. We have a lot of people in Ataccama who are fairly technical and would not shy away from writing a short script in python, but the barrier of setting everything up, making all the libraries work, scraping the relevant data etc — these things would complicate the hackathon too much.</p><h3>Getting people excited and letting them learn at the same time</h3><p>As I mentioned before, I wanted as many people to join as possible. And many were not sure if this is really an event for them. We did many shorter sessions and repeated slack announcements explaining why we believe this Hackathon is for everyone. There were 3 specific noteworthy events that we conducted:</p><ol><li>The introduction of the Hackathon included a short session delivered by me about LLMs, an introduction what it is, how it relates to our business and why I think its such a big deal now. Kinda similar to my previous two posts — <a href="https://medium.com/@roman.kucera/what-is-gpt-and-why-it-became-so-popular-this-year-4bf3d68e3c7a">What is GPT and why it became so popular this year?</a> and <a href="https://medium.com/@roman.kucera/can-gpt-help-with-data-management-3f65fdf5e62c">Can GPT help with data management? </a>— with a bit more of my ideas where this might lead, lets call it food for thought.</li><li>An hour long session with GPT tips and tricks — showing people (including the non-technical ones) how to play with GPT in Azure OpenAI Studio and showing examples of prompt engineering techniques, how a small change in a prompt can make the difference between anecdotical failures or the magical wow effect that LLMs just work. In the end I did this session myself even though I did not consider myself an expert — but I knew the audience and I knew what kind of a message I wanted to deliver … and it was a great excuse to learn these things more properly while preparing for the session 🙂</li><li>About 90 minute technical workshop where people would use the prepared jupyter notebook to learn how to use OpenAI APIs as well as some more advanced techniques — like using LangChain library to use LLMs together with local knowledgebases, etc.. I asked some of our good friends from <a href="http://blindspot.ai/">Blindspot.ai</a>, our sister company that specializes in AI solutions, to run this workshop — so we actually had people with as much experience as possible in this particular area (which for LangChain specifically meant probably just couple more weeks of experience compared to us, considering how young these technologies are, but still 🙂 ).</li></ol><p>Apart from these classes, we provided a lot of learning materials as well. But from what I have seen I think the live sessions had much bigger impact than anything else.</p><blockquote><strong>Tip 5:</strong> The feedback to all three of these sessions was overwhelmingly positive. To the date, people come to me and ask me if we can continue doing these kinds of sessions regularly, even without any hackathon. I think it is absolutely worth it planning these things seriously. The content we had was good, but it didnt feel like anything too special — we probably could have just post links to online courses that were of similar quality. But many people did not want to spend time reading guides or watching online videos, yet they enjoyed the live session very much. The personal touch is really important, the hackathon experience is very much an emotional one.</blockquote><h3>Mentors</h3><p>Its important that there is always someone to help teams if they get stuck, or just to give them feedback on their work. We did this through a couple of mentors — external ones for the technical help with generative AI, internal ones (including myself) for the overall help with the usecase, idea brainstorming, etc.</p><p>During the hackathon itself, I tried to walk around the teams (including the remote ones) at least 2x every day to see how they are doing and if there is anything we can do to help them. They could also ask us for help at any time in a dedicated slack channel.</p><p>Some of the less technical teams found it really helpful that there was always someone to help them with problems in their code. And the more technically capable teams just picked our brain on what could be improved for their final presentation.</p><h3>Team Formation</h3><p>Organizing team formation in a hybrid event on such a short notice when one of our main goals was to promote cross team collaboration seemed like a big challenge. Fortunately Alzbeta from the organization team saved the day (Thank you!) by preparing a Miro board for this excercise. People would fill in their details like what they bring to the table, when are they available, etc. Then some of them would write their hackathon ideas in so called campfire areas, while others would just put their card next to the idea they liked the most … people could discuss the idea and eventually agree to form a team. Not everybody joined this excercise, but we managed to form 8 teams this way and then chase the remaining people individually and find best arrangements for them. In the end, we had 10 teams with 42 hackers in total participating in the event.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mREBSTYeCFf2Irnm-VSOoQ.png" /><figcaption>The team formation Miro board (this is a very large picture, zoom in if you want to read the exercise details)</figcaption></figure><h3>Other organisational aspects</h3><p>There were other things we needed to consider for the hackathon. We had a backup plan if our Azure OpenAI Service resource went down, we had an IT support on standby if any of our infrastructure failed over the weekend, we had the security team issue a statement on what data we can or cannot use with Azure OpenAI Services (and they were ready to address any ad-hoc questions during the event). We had some light refreshment available all the time in the main offices. Even though we considered hackathon to be a bit of a punk event, we did our best to have everyone enjoy the event with as little problems as possible.</p><blockquote><strong>TIP 6:</strong> Don’t overthink it, just start hacking already 🙂. Be ready to face issues when they arise, but don’t fuss about it too much… I guess this applies not only to hackathons, but I felt this was very relevant for our planning. We like our events perfect. It is OK if the hackathon is a bit messy.</blockquote><h3>Phase 2: The event itself</h3><p>I do not want to spend too much time on what the individual teams did. You can read our <a href="https://www.ataccama.com/blog/hackathon-2023-gen-ai-for-data-management">official hackathon post</a>, which describes some of the results.</p><p>Only a few teams ended up working over the full weekend in the end. That being said, even the teams that worked only on Friday produced great results — and for some of them, I was genuinely surprised how much they could deliver in just a single day.</p><p>One of the teams did a full blown agent implementation with over 10 tools at the agents disposal, integrating many parts of our Ataccama ONE product with the language model. In this process they found out that the LangChains support for agents was really crappy at that point in time and they actually had to rewrite the implementation completely on their own to make it actually work well.</p><p>The overall vibe of the event was great. Everyone has managed to achieve something and from what I could tell, everyone enjoyed the process.</p><h3>Phase 3: Evaluation</h3><p>I have seen non-competitive hackathons before, but I do believe its much better to have some evaluation at the end. People are competitive in nature and it just feels right to reward the best team. The simplest way is to just have each team present their outcomes to a jury, which will then pick a winner.</p><p>We decided to rate teams according to how well they represented Ataccama values. Defining company values would be a topic for a whole another post, but very loosely described, we rated each of the teams on these 5 scales:</p><ol><li>How much business potential did the team show? Were they listening to real customer needs, or just playing around?</li><li>Was the team clear and concise? Were they positive? Did they show respect?</li><li>Did the team surprise us? Did they challenge assumptions and bring new perspectives?</li><li>Did the team actually work as a team? Did they manage to bring some value from the cross-departmental collaboration?</li><li>Did the team have fun, did they learn something new, did they overcome challenges?</li></ol><p>Our Jury consisted of 10 people from the upper management of the company and everyone would assign a score to each team on each of these scales. Overall, it was a bit messy and I would probably prefer a simpler solution next time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jr3rQEzqJCTU18_uvBFqsQ.png" /><figcaption>A score card that each jury member had to fill</figcaption></figure><p>Interestingly, many people asked about specifics how the winner is going to be picked and in general they were not very satisfied with my vague description “the jury will pick the winner”. Maybe I am weird, but I really wanted people to show me what they thing is a worthy outcome of the hackathon, not to try to play the system and optimise for some artificial metric.</p><p>The good thing is that in the end, we could see a fair spread among the values, for each value there were some teams that shined and some teams that did bellow average. We decided to add the scores from all categories together to get the overall winner, but we also announced a secondary prize for each of the categories individually.</p><p>Members of the winning teams could then pick a small prize. In the end the options included Raspberry Pi, Earbuds (AirPods or Sennheiser Momentum) or a backpack. Most people chose the Raspberry Pi, which goes kinda well with the hackathon spirit I guess. I originally really wanted some generative AI prizes, like ChatGPT+ subscription, Midjourney subscription, or something like that. Turns out these services are not ready for business customers, especially not ones from Europe. It is a no-go for us as a company to put a company credit card into someones personal account, without any control over it. And OpenAI even specifically mentions it does not accept prepaid cards. Oh well, what can we do…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DJYJOd8ThKeiT8ExSb5LDA.jpeg" /><figcaption>Unboxing of the most popular hackathon prize</figcaption></figure><h3>Costs</h3><p>We were slightly worried about the cost control — a lot of the teams ended up using GPT-4, some even the 32k context version, and some were prompting the language model in bulk or somehow in an automated fashion. Microsoft does not offer any way to enforce any limits on the Azure OpenAI API. There is some limited way to set a per-minute quota today, but this wasnt available back then and it wouldnt be enough anyway. The only way to have any control over how much individual teams spend would be to create a proxy in front of the API, authenticate the users and measure the consumption on the proxy. This seemed like too much of a complexity for the hackathon, so we just asked all the teams to be reasonable and hoped for the best. In the end, the total Azure bill was roughly 840 euro for the whole event, much less than we expected, so this worked out OK. Other than that, we spent a bit on food and of course on all the prizes, but overall, this was a pretty low cost event.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-zubZ0eP2FDuIjz-NDCZmw.png" /><figcaption>Azure OpenAI costs details for the hackathon period</figcaption></figure><h3>Conclusion — Was it all worth it?</h3><p>It was absolutely worth it.</p><p>If you are in doubt whether to do a hackathon or not, I would definitely encourage you to try. It is a lot of work for sure, you might feel some pain along the way, but it feels very rewarding in the end.</p><p>All of the hackathons I ever participated in were great. This particular event was amazing and I am very glad we have done it!</p><p>I think we did meet the expectation of everyone involved, we did learn a lot, we had fun. As for the technical outcomes, the demos were great, but obviously that is what they were — demos. Keep this in mind, you need to have reasonable expectations.</p><p>Now, 2 months later, we are finally getting some of these projects ready for real usage. Of course these production ready implementations are more just loosely inspired by what hackathon teams produced, but still, this is super important for people to see at least some of the hackathon projects make it in the end.</p><p>If you have any questions on what we did, or if you have any tips on what worked for you and what we should try next time, please leave a comment!</p><p>Also look out for more posts in this LLMs in Data Management series. Now that the hackathon is over and we had some time to process all the outcomes, I will try to finally focus on what impact we see LLMs will have on data management in near future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23c6846d5127" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can GPT Help with Data Management?]]></title>
            <link>https://medium.com/@roman.kucera/can-gpt-help-with-data-management-3f65fdf5e62c?source=rss-68a794fa3f11------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f65fdf5e62c</guid>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Roman Kucera]]></dc:creator>
            <pubDate>Fri, 25 Aug 2023 16:18:30 GMT</pubDate>
            <atom:updated>2023-08-29T12:29:50.885Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6wPMMRF69mW0a8dglzMzlA.jpeg" /></figure><p>For me, the last few months have been about generative AI and, possibly, for many of you as well. After working on adopting large language models at Ataccama, I decided to share a little about our journey. This will be a series of posts — covering different steps along the way.</p><h3>How we got started with GPT at Ataccama</h3><p>We started talking about generative AI and large language models more seriously around March this year. A few people from our AI community warned me this AI wave is coming much sooner than that — GPT-3 has been around for years now — so we were monitoring the situation but it wasn’t really the main focus before.</p><blockquote><em>If you do not know the GPT story very well, you can read </em><a href="https://medium.com/@roman.kucera/what-is-gpt-and-why-it-became-so-popular-this-year-4bf3d68e3c7a"><em>my short summary here.</em></a></blockquote><p>With the release of ChatGPT, and later on GPT-4, it became obvious there was a potential revolution brewing. We had to react. Now, I believe generative AI is a complete game-changer that impacts and will continue to impact pretty much everyone.</p><p>When it comes to data management, you can try some simple questions in the basic ChatGPT interface to test its skills. Lets look at couple of examples. Even with very basic prompts, you can see that the tool shows the capacity to understand data.</p><p>Here, it is describing a data table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/0*DYIZBpEgru9LYHp0" /><figcaption>ChatGPT describing one of our tutorial csv files</figcaption></figure><p>Here, you can see it suggesting business definitions of DQ rules for the same table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/0*5CMqCoPoF-qjvXjZ" /><figcaption>Asking ChatGPT to define validation rules</figcaption></figure><p>And it can also write a statement in SQL to evaluate such rules:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/822/0*Wc-T00PLxZxnOAv0" /><figcaption>Once the rules are defined, GPT can implement them in various languages, including SQL</figcaption></figure><p>As a proof of concept, these prompts work really well. However, there are still many issues you’d face if you tried to use this basic approach at scale on your data. The first problems you’d encounter are related to the model’s capabilities — mainly its limited knowledge, limited context, and tendency to hallucinate answers that sound plausible but are completely made up.</p><p>While these challenges seem significant, there are reasonable solutions for each of them. You just have to engineer your way around them. You can provide relevant context, and use concepts such as semantic search to pre-filter relevant facts and let the LLM use them to answer. You can also teach the AI to use external tools to mitigate its weaknesses. In the end, you can combine techniques like these to make the AI do whatever you want — including referencing its sources, verifying its own work, or iteratively working towards its ultimate goal.</p><p>OpenAI released ChatGPT+ with a browsing mode and plugins in May. Below is a great comparison of how the simple usage of a tool can overcome basic LLM limitations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/0*ilaepAvVnnEsz5Y2" /><figcaption>Asking plain GPT-4 model a fact based question that requires recent knowledge</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/0*2FEfN7bIcmgy1EsP" /><figcaption>Asking the same question in the browsing mode — the information is looked up online</figcaption></figure><p>In July OpenAI also released the Code Interpreter, which is a great showcase of how to overcome many of the limitations of a standalone large language model. This wasn’t available when we were deciding how to start using large language models, but it is the simplest way to demonstrate some of the principles. Let’s take a look at an example of how it can tackle a simple data management problem:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hqN9M-E_DsMeyjkbnSYM7A.png" /><figcaption>GPT-4 with Code Interpreter doing some autonomous work</figcaption></figure><p>Note that even though the output is long and with many many steps, there was no back and forth between me and the model. My input throughout the whole process was just one sentence in the beginning. What we can see is a nice iterative approach, essentially doing these steps:</p><ol><li>Ask GPT-4 what needs to be done</li><li>Unless the answer is obvious already, GPT-4 writes python code that will help to get the desired answer</li><li>Python code gets executed</li><li>Return to step 1</li></ol><p>This approach is fairly generic and applicable to many fields. And it can of course be combined with other techniques, like the browsing mentioned above, or with some local knowledge base.</p><p>So all problems solved? AI can do data management now? Not quite yet. Hidden between the lines is the cumbersome engineering effort required to come up with these solutions and then to polish them. If you take a closer look at what the code interpreter did in the example, it was great considering how little effort it took and that the tool was not even primarily developed for these usecases. But to get valuable outputs in real life, we would need to do quite a bit more work.</p><p>What is absolutely amazing is that now we have all the building blocks to actually do complex things automatically. And when done correctly, it seems we can get results that feel like magic. But it’s important to keep in mind that the common message “AI solves everything” is a bit too optimistic at this point. AI will help us. It’s a great tool. But we are the ones who have to solve the problem, at least for the time being.</p><blockquote><em>Along these lines Honeycomb published a great post about this:</em><a href="https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm"> <strong>All the Hard Stuff Nobody Talks About when Building Products with LLM</strong></a>.</blockquote><p>What does all this mean for Ataccama?</p><p>The conclusion here is that we believe LLMs will have a tremendous impact on our business, and that it will involve a lot of work. So where do we start?</p><p>As the next step, we organised an internal hackathon around LLMs. I guess this is not very original, I have seen many companies doing the same thing, but still, I believe it was a good and important step on our journey. You can <a href="https://medium.com/@roman.kucera/lessons-learned-from-organising-an-internal-openai-hackathon-23c6846d5127">read more about it in the next post</a>!</p><p>To stay updated on what we are doing, follow me, and/or join our <a href="https://www.ataccama.com/events/gen-ai-transforming-the-data-landscape-fireside-chat-216">Generative AI event</a> to engage with thought leaders across the industry.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f65fdf5e62c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is GPT and why it became so popular this year?]]></title>
            <link>https://medium.com/@roman.kucera/what-is-gpt-and-why-it-became-so-popular-this-year-4bf3d68e3c7a?source=rss-68a794fa3f11------2</link>
            <guid isPermaLink="false">https://medium.com/p/4bf3d68e3c7a</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Roman Kucera]]></dc:creator>
            <pubDate>Fri, 25 Aug 2023 16:11:13 GMT</pubDate>
            <atom:updated>2023-08-25T16:18:59.015Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j6rlqVwJ88ivAOvvz4yJ1Q.jpeg" /></figure><p>For me, the last few months have been about generative AI and, possibly, for many of you as well. After working on adopting large language models at Ataccama, I decided to share a little about our journey. This will be a series of posts — covering different steps along the way.</p><p>Let’s start with a short recap of relevant terminology and what GPT is.</p><p><strong>GPT</strong> stands for <strong>Generative Pre-trained Transformer</strong>.</p><p>“<strong>Transformer”</strong> is a type of a neural network architecture that <a href="https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html">Google introduced in 2017</a>. Its a rather simple neural network compared to many other architectures we know, and is well-suited to run on currently available hardware. This means it is feasible to make a neural network very large and still be able to handle the associated computational complexity. GPT-3 (<a href="https://openai.com/blog/openai-api">released in 2020</a>), for example, is a transformer with 175 billion parameters. This is one of the reasons we call these models “large language models.”</p><p>“<strong>Pre-trained”</strong> means that a model has been trained on large volumes of data without any specific task in mind. It uses text to learn dependencies between words and other features of language. In the case of GPT-3, this includes a large collection of books, the entire English Wikipedia, and a fairly large subset of the Internet — altogether in the order of hundreds of millions of words. This is the second reason why we call these <strong>large</strong> LMs.</p><p><strong>“Generative</strong>” means that the model can produce new text. When given an input, it generates text that is likely to follow based on all of the language understanding the model has learned. What surprises many people is that the generated output is actually original. The model doesn’t just output pieces of what it has seen before; rather, it has learned the fundamental structure of the language, and can produce completely new content. What’s even more interesting is that it seems to mimic reasoning. When you think about it, this makes sense. Humans also use language to reason. Try to think something through without using words. Difficult, isn’t it? The LLM model has learned to use language and it can even simulate the thinking process.</p><p>I have used GPT-3 in the explanations above mainly because we still do not know much about GPT-4 yet (OpenAI stopped being so open at some point).</p><h3>Meet ChatGPT</h3><p>As you might have noticed, this technology is not really that new. So why is all this hype coming now, in 2023?</p><p>On November 30, 2022, Open AI introduced ChatGPT. Its a fine-tuned version of the model, together with a chat based interface that just about anyone can use. It immediately became <a href="https://www.makeuseof.com/how-chatgpt-became-fastest-growing-app/">the fastest growing application ever made</a>.</p><h3>What does it mean to be “fine-tuned?”</h3><p>The plain “vanilla” GPT-3 model (also called the foundation model) is not that nice to use. It knows the structure of a language, but does not necessarily listen to instructions or answer questions. It simply knows what is likely to follow after any given input from the language perspective (and more specifically, from the training data perspective).</p><p>If you ask GPT-3 a question, it’s possible that, from a language perspective, replying with another similar question is reasonable. It might also be acceptable from a language perspective to tell you not to bother the AI, and to Google your question instead. Even telling you a wrong answer is something that the AI could have picked up from its sources. Lying might be morally wrong, but from language perspective, from what the model could have learned from the samples, its still a valid answer. Many of these scenarios are completely fine from a generic language model perspective. They’re just not very helpful to the end user in most cases.</p><p>Fine-tuning means adjusting the language model to work well for a specific task. In the case of ChatGPT, its task is to be a helpful assistant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hgwcBUWx01ImSL6D_qasMA.png" /><figcaption>Asking ChatGPT for its purpose</figcaption></figure><p>Such fine-tuning is typically done by giving examples of prompts and good responses, and then adjusting the weights of the model so that it is more likely to produce such good responses. But giving the model enough examples to really improve its capacity can be tricky. What OpenAI did here is something called Reinforcement Learning from Human Feedback (RLHF).</p><p>The basic idea behind RLHF is that it is much easier rate answers (out of a few options, which answers are better than others) than actually write a good answer. You start with a base foundation model, give it questions, let it generate multiple answers, and then rank these answers manually — this is the Human Feedback part. When you collect enough of these examples, you can fine tune the model to do the ranking at a performance level similar to that of humans. Once this step is done, you can use a reinforcement learning technique I <a href="https://medium.com/p/37c5d680e5a7">blogged about a couple of years ago</a>. You let another instance of the model produce answers to questions, and the fine-tuned ranking model ranks them. Then you let it run for some time, fine tuning the model along the way to produce better and better answers to your questions.</p><p>It sounds easy enough. The reality, of course, is much more messy. Long story short, OpenAI came out with a version of their model that is much more helpful than the original base model. And at the same time, they made it available to the general public, for free, and in an interface that is easy to use even for non-technical people. Technologically, the jump from the 2020 version of the model to ChatGPT was not that big. It’s this fine-tuning that started the AI fever of 2023. It brought more attention, brought more researchers to this field, brought more funding — and now we see possible applications everywhere.</p><p>To give you an idea of how fast this whole space is moving, check out <a href="https://github.com/hollobit/GenAI_LLM_timeline">this GitHub project</a> that shows a timeline of all important milestones, including research papers, interesting projects, and important product releases that are relevant to large language models. For May 2023, it shows 324 such events, equalling more than ten per day on average. Its getting really hard to keep up.</p><p>(update 2 months after I have written the first version of this article — in July this was only 232 events … the question is whether the hype is cooling down, or this is the summer holiday effect, or just the maintainer of this repository became more selective)</p><p>Wondering what all of this means for data management? Read on the next part in the series, <a href="https://medium.com/@roman.kucera/can-gpt-help-with-data-management-3f65fdf5e62c">Can GPT help with data management?</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4bf3d68e3c7a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The truth behind Facebook AI inventing a new language]]></title>
            <link>https://medium.com/data-science/the-truth-behind-facebook-ai-inventing-a-new-language-37c5d680e5a7?source=rss-68a794fa3f11------2</link>
            <guid isPermaLink="false">https://medium.com/p/37c5d680e5a7</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[language]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[facebook]]></category>
            <dc:creator><![CDATA[Roman Kucera]]></dc:creator>
            <pubDate>Mon, 07 Aug 2017 15:18:55 GMT</pubDate>
            <atom:updated>2017-08-15T14:24:50.355Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*wTDT5wqWWNLCQp9x.jpg" /></figure><p>There have been so many articles published about <a href="http://www.telegraph.co.uk/technology/2017/08/01/facebook-shuts-robots-invent-language/">Facebook shutting down its robots after they developed their own language</a>. The media is just loving these clickbait titles. Some of these articles would let you believe that this was a very close call — that scientists at Facebook barely shut down the AI before it could take over the world. Fortunately, there are still sane people out there, so there have been quite a few articles explaining why all of the doomsday talk is complete nonsense (like the ones published by <a href="http://www.snopes.com/facebook-ai-developed-own-language/">Snopes</a> or <a href="https://www.cnbc.com/2017/08/01/facebook-ai-experiment-did-not-end-because-bots-invented-own-language.html">CNBC</a>). Even some of the media that originally offered a very scandalous version of this event eventually edited the content to be less dramatic (like <a href="http://www.independent.co.uk/life-style/gadgets-and-tech/news/facebook-artificial-intelligence-ai-chatbot-new-language-research-openai-google-a7869706.html">The Independent</a> for example).</p><p>The problem is that false but catchy news is much easier to spread than anything else. I also find it curious that none of the articles actually explained what happened in terms that people would understand. I tried to explain the situation to some of my friends and eventually decided that it was worth writing down. Maybe it will help people sleep without thinking of SKYNET. However, my goal is to educate — to show how so-called AI works — not to take sides.</p><h4>Facebook AI Research (FAIR)</h4><p>So what was Facebook actually doing? And how did the robots “almost become sentient”? The whole project is well-documented and available to the general public. Anyone can actually download and run this AI, as well as observe the new language on their own. Just please be careful and shut it down in time like the Facebook engineers did.</p><p>The system tries to simulate dialog and negotiation. The so-called robot is given a set of items (consisting of books, hats, and balls) and some preferences for which items it wants more than others. Then it is supposed to negotiate with its counterparty, be it a human or another robot, about how to split the treasure among themselves.</p><p>The research was published in June, including all code and training data used for the experiment. If you are interested in more details, read the official <a href="https://code.facebook.com/posts/1686672014972296/deal-or-no-deal-training-ai-bots-to-negotiate/">article</a> or just get the code from <a href="https://github.com/facebookresearch/end-to-end-negotiator">github</a>.</p><p>So how does it work in simple terms?</p><h4><strong>Machine Learning</strong></h4><p>I will not bother you with all the technical details, but it is important to understand some basic principles about how this technology works.</p><p>When developing a robot like this, you start with something called a “training data set”. This consists of well-described examples of the behavior that the robot is trying to simulate. In the particular case of the Facebook negotiation chat bot, you give it examples of negotiation dialogs with the whole situation properly annotated — what the initial state was, the preferences of the negotiator, what was said, what the result was, etc. The program analyzes all these examples, extracts some features of each dialog, and assigns a number to these features, representing how often dialogs with that feature ended in positive results for the negotiator. To better imagine what a “feature” is, think words, phrases, and sentences. In reality it is more complicated than that, but it is good enough to get the principle.</p><p>To be more specific, if the robot wants hats, the phrase “You can have all the hats” will have a really low score because this sentence ended with a bad result in every scenario from the training data— the negotiator did not get what he wanted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/0*s4ViygzWkGH7ny8-." /><figcaption>Diagram representing a sample dialog evaluation (<a href="https://arxiv.org/abs/1706.05125">Deal or No Deal? End-to-End Learning for Negotiation Dialogues</a>, 2017)</figcaption></figure><p>This will basically get you version zero of your AI. It now knows which sentences are more likely to get a good deal from the negotiation. You can use it to start a dialog. It will try to maximize the probability of a positive outcome based on the numbers gathered during the training phase. The term AI feels kinda weird here — it is very artificial, but not very intelligent. It does not understand the meaning of what it is saying. It has a very limited set of dialogs to relate to, and it just picks some words or phrases based on probabilities calculated from those historical dialogs. It cannot do anything else. It just calculates the probability of getting the desired amount of hats, balls or books, and based on that it writes something on the screen.</p><p>The next step is using a technique called reinforcement learning. As our ability to provide well-annotated training data is fairly limited, we need another way for this AI to learn. One of the common approaches is to let the AI run a simulation, and learn from its own results. Before I explain what this meant for the negotiation robot case, let me sidestep to a different AI.</p><h4>AlphaGo</h4><p>Google Deepmind AlphaGo is a program that you may have heard about last year. It was the first AI to beat a professional Go player. And it is a perfect example of reinforcement learning in action.</p><p>AlphaGo started learning from real games played by real people. It analyzed and scored each possible move based on this knowledge. This alone made AlphaGo capable of playing, albeit very poorly — it did not understand the game, but it had a way to score the moves based on previously analyzed games.</p><p>But, Go is fairly easy to simulate. We have an exact set of rules and we have a very good goal for the AI — to win the game. So we can just create two instances of such an AI and let it play against itself. Since we have a lot of computing power available, it can easily play millions of games to train, many more than any human ever could. It then updates the probabilities of a win for each move based on all of these simulated results, getting better and better at scoring the moves well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/0*J2HH4TDqGCuKbufS.png" /><figcaption>The famous match of AlphaGo vs Lee Se-dol. The long reinforcement learning period is paying off.</figcaption></figure><p>Again, I am simplifying the concept. If you want to learn more about AlphaGo, I can recommend <a href="https://www.tastehit.com/blog/google-deepmind-alphago-how-it-works/">this article from Christopher Burger</a>. I just want you to take one thing away from this example. Reinforcement learning work really, really well (as proven by AlphaGo and many others) if we can satisfy three conditions:</p><ol><li>A well-defined space of options for the AI. In the case of AlphaGo, it can only play valid Go moves.</li><li>A good way to score the outcome. In the case of AlphaGo, a win is good, a loss is bad.</li><li>A good way to simulate the situation and let the AI learn. In this case, we can just let the AI play Go against itself (preferably against different versions of itself as it is easier to learn something new if your opponent plays differently than you).</li></ol><h4>Back to Facebook</h4><p>So let’s look back at the original topic. Facebook was trying to create a robot that could negotiate. How well does it fit the three conditions above? There is a great way to score the outcome as there is a value assigned to each item that is part of the negotiation. We have a good way to simulate the situation and let the AI learn. That is exactly what Facebook did — let two instances of the robot negotiate with each other. But the last point is slightly problematic.</p><p>Unlike with the game of Go, there is no easy definition of the English language. The original training data set was in English, but the extracted features were just words and phrases, and the robot was just putting them together based on the numerical representation of how likely they were going to help get the desired outcome.</p><p>Two robots that do not actually understand English ended up talking to one another and learning from one another. The only measure of their success was how well they distributed books, hats, and balls. The only rule to follow was to put words on the screen. They started talking more or less in English, but they were learning from their own mistakes, without knowing they were actually mistakes — led to the desired outcome in the form of hats, books, and balls. Some words got lost because according to the numbers, they did not contribute to the negotiation outcome. Some got multiplied. If saying “I want” improves the chance of getting something, then why shouldn&#39;t we say it multiple times, right? And once this works, the AI will take it as confirmation that this is a good strategy.</p><p>If it was training against a human being, this would probably not be such a big problem, as the other side would be using proper language. There would be a different problem though. It would be difficult to run a large enough number of simulations to train the AI.</p><p>As Facebook engineers noted, it could have worked better if the scoring function had also included a language check, rather than only the total value of items received after the negotiation. But it didn’t. The fact that the language degenerated is neither surprising nor interesting in any way. It happens to every scientist working on these types of problems, and I am sure Facebook engineers actually expected that result. They just turned off the simulation once it degenerated too much, after many iterations, and after it stopped providing useful results.</p><p>By the way, if you read the report or the published paper, apart from the gibberish conversation that was shared all over the internet, there were actually many good results as well. The experiment worked as intended, and I would say was pretty successful overall.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/0*aztcWn318h13Mq7t.jpg" /><figcaption>This is not a new “more efficient” language that only AI understands as some journalists would lead you to believe. It is just a degenerate form of English after too many rounds of reinforcement learning.</figcaption></figure><h4>In the end, it’s nothing surprising</h4><p>Nothing scary, shocking or even note-worthy happened. Just a regular day with a regular scientific experiment. And then all hell broke loose as wanna-be journalists posted one doomsday article after another. Shutting down a chat bot when it stops showing a reasonable outcome is about as ominous as changing a faulty light bulb.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37c5d680e5a7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/the-truth-behind-facebook-ai-inventing-a-new-language-37c5d680e5a7">The truth behind Facebook AI inventing a new language</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>