Increase your business’ chances of survival by arming yourself with the best information possible.

Are you worried about the long term health of your business?

If so, you’re not alone. As the initial shock subsides and we find the mental space to act strategically, we still find ourselves strapped with one main problem: we don’t know what will happen.

It’s hard to plan for what you don’t know

DareData, along with the rest of the business community, have now lived through the first month of lockdown in the western world. We’ve got some idea of what happens to our sales in the worst case scenario which we can and should start to leverage. However, there are still many things that we don’t know. According to many experts (see our previous post for more details), the following outcomes are being viewed as very possible, if not likely:

• Extended current lockdown
• Multiple lockdowns
• Required or voluntary social distancing until therapies are developed
• A vaccine could be years away

So what can you do?

Even if you don’t know what will happen, you can and should prepare yourself with data showing what will happen to your revenue in each of the projected cases. Armed with this data, it could help with the following:

• Asking your government for financial help
• Adjusting your short, medium, and long term business strategies to ensure survival

Familiarizing yourself with the expected impact on your sales under different scenarios will allow you to put informed plans in place for each one of them. This way, when one of the scenarios starts to become true, you’ve got a well-thought-out plan that can be activated, rather than having to rush the plan development and implementation all at once.

This type of stress testing is well-studied though it is not very widespread amongst small and medium size businesses.

How we can help you

Send us your sales data (sales.forecasting@daredata.engineering) and we will deliver weekly reports with forecasts adjusted for some or all the scenarios that McKinsey has developed in this excellent report. Some of these scenarios are as follows and are are done using obfuscated data from a local restaurant in Lisbon, Portugal.

All of your data will be kept anonymous, encrypted, and will not be shared with anyone who is not directly contracted by DareData. We will gladly service deletion requests immediately.

Scenario 1: Virus contained and mid to slow recovery

Public health authorities are able to contain the virus until a vaccine is out. The economic rebound will only happen in two quarters and by the end of the year economies are still struggling but picking up.

This scenario is characterized by:

• One lockdown period of 8/10 weeks;
• An initial drop of x percent (we get this from your data).
• Some businesses die and ⅓ of them are replaced by new customers.
• The economic recovery happens at a rate of 3% per month and sales pickup on Q42020.

Scenario 2: Virus contained and slow economic recovery

Public health authorities are able to contain the virus until a vaccine is out. The economic rebound will happen slowly and the economy will only pick up after Q1 2021.

This scenario is characterized by:

• One lockdown period of 8/10 weeks;
• An initial drop of x percent (we can get this from your data).
• Some businesses die and 1/10 of them are replaced by new customers.
• The economic recovery happens at a rate of 1% per month and sales pickup after Q12021.

Scenario 3: Virus not contained and slow economic recovery

Public health authorities are not able to contain the virus until a vaccine is out. The economic rebound will happen slowly and the economy will only pick up after 2021.

This scenario is characterized by:

• Two lockdown periods of 15 weeks;
• An initial drop of x percent (we get this from your data).
• Some businesses die and almost none are replaced.
• The economic recovery happens at a rate of 0.5% per month and sales pickup after 2021.

Our method

For each of these scenarios, we will use the following tool that we’ve developed along with your dataset(s) to come up with adjusted sales forecasts. We hope to be releasing a web app in the future so that people can maintain their own forecasts.

Is this really free?

We will provide your first three weekly reports for free. With this, you should derive significant value. The reports that are delivered take into account policy and scientific data as it becomes available. It is up to you to decide whether or not you will continue to need them after the first 3 weeks. Pricing will be announced soon.

Free sales forecasts was originally published in DareData Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

Are you worried about the long term health of your business?

If so, you’re not alone. As the initial shock subsides and we find the mental space to act strategically, we still find ourselves strapped with one main problem: we don’t know what will happen.

It’s hard to plan for what you don’t know

DareData, along with the rest of the business community, have now lived through the first month of lockdown in the western world. We’ve got some idea of what happens to our sales in the worst case scenario which we can and should start to leverage. However, there are still many things that we don’t know. According to many experts (see our previous post for more details), the following outcomes are being viewed as very possible, if not likely:

• Extended current lockdown
• Multiple lockdowns
• Required or voluntary social distancing until therapies are developed
• A vaccine could be years away

So what can you do?

Even if you don’t know what will happen, you can and should prepare yourself with data showing what will happen to your revenue in each of the projected cases. Armed with this data, it could help with the following:

• Asking your government for financial help
• Adjusting your short, medium, and long term business strategies to ensure survival

Familiarizing yourself with the expected impact on your sales under different scenarios will allow you to put informed plans in place for each one of them. This way, when one of the scenarios starts to become true, you’ve got a well-thought-out plan that can be activated, rather than having to rush the plan development and implementation all at once.

This type of stress testing is well-studied though it is not very widespread amongst small and medium size businesses.

How we can help you

Send us your sales data (sales.forecasting@daredata.engineering) and we will deliver weekly reports with forecasts adjusted for some or all the scenarios that McKinsey has developed in this excellent report. Some of these scenarios are as follows and are are done using obfuscated data from a local restaurant in Lisbon, Portugal.

All of your data will be kept anonymous, encrypted, and will not be shared with anyone who is not directly contracted by DareData. We will gladly service deletion requests immediately.

Scenario 1: Virus contained and mid to slow recovery

Public health authorities are able to contain the virus until a vaccine is out. The economic rebound will only happen in two quarters and by the end of the year economies are still struggling but picking up.

This scenario is characterized by:

• One lockdown period of 8/10 weeks;
• An initial drop of x percent (we get this from your data).
• Some businesses die and ⅓ of them are replaced by new customers.
• The economic recovery happens at a rate of 3% per month and sales pickup on Q42020.

Scenario 2: Virus contained and slow economic recovery

Public health authorities are able to contain the virus until a vaccine is out. The economic rebound will happen slowly and the economy will only pick up after Q1 2021.

This scenario is characterized by:

• One lockdown period of 8/10 weeks;
• An initial drop of x percent (we can get this from your data).
• Some businesses die and 1/10 of them are replaced by new customers.
• The economic recovery happens at a rate of 1% per month and sales pickup after Q12021.

Scenario 3: Virus not contained and slow economic recovery

Public health authorities are not able to contain the virus until a vaccine is out. The economic rebound will happen slowly and the economy will only pick up after 2021.

This scenario is characterized by:

• Two lockdown periods of 15 weeks;
• An initial drop of x percent (we get this from your data).
• Some businesses die and almost none are replaced.
• The economic recovery happens at a rate of 0.5% per month and sales pickup after 2021.

Our method

For each of these scenarios, we will use the following tool that we’ve developed along with your dataset(s) to come up with adjusted sales forecasts. We hope to be releasing a web app in the future so that people can maintain their own forecasts.

Is this really free?

We will provide your first three weekly reports for free. With this, you should derive significant value. The reports that are delivered take into account policy and scientific data as it becomes available. It is up to you to decide whether or not you will continue to need them after the first 3 weeks. Pricing will be announced soon.

Because, like these good medical professionals, I’m scared.

Why am I scared?

Because the human condition has left us ill-equipped for a crisis of this nature.

What is the nature of this threat that scares me?

It is a threat that only needs one weak link to wreak havoc.

Why does the human condition combined with the nature of this threat scare me?

The human condition is all about fighting the good fight between short and long term gratification. We are hard-wired to want things that make us feel good in the short term. We like to overeat, fuck, fight, drink, do drugs, overwork, and all kinds of other risky behavior because evolution has had billions of years to craft us this way. And the crafting makes sense: they have had a huge hand in making us the dominant species on our planet.

This very sloppy (but effective) process of evolution has created a very diverse population of people that have all kinds of different strengths and weaknesses. Some are good long term planners. Some are very creative. Some are more productive. Some are more driven and focused. Some are also more sloppy. Still more are very greedy. However, we all have one thing in common: the human condition.

When you have a whole society in which our populations are so diverse in their strengths and weaknesses, it is usually a good thing. All we need to do is have some portion of the population that is willing to act in good faith and it makes even the most difficult challenges doable.

This one is different. A single weak link in a huge population is enough to wreak havoc on the lives of thousands. Literally thousands. The spreading rate of COVID-19 looks to be very high. High enough that a single person is enough to start an outbreak that, if no measures are taken, can lead to tens of thousands of infections.

Fighting this threat is, in many ways, fighting our desire for short term gratification. We want to go out to eat. We want to go to the gym. We want to go to the party. We want to go to the bar. We are social animals! We are creatures of habit! Of course we want all of this! And when all of the guidelines from all of the experts tell us we cannot do any of these things, the challenge becomes a fight against instant gratification.

So this essentially boils down to a fight against instant gratification against a threat that only requires a single weak link to bring down entire economies. The prognosis is not good.

An erosion of trust in science has left us vulnerable

The sheer amount of disinformation circulating the web has left people distrustful in anything that resembles authority. While we should all definitely be wary of authority, it seems like many of us have lost the ability to distinguish between scientific authority and political authority. Scientific authority is generally accurate. Here’s a few facebook posts of a particularly conspiracy-laden facebook friend of mine from the beginning days of the outbreak in the states.

This one is basically calling you a pussy if you take the threat seriously:

And this one questions whether or not there was actually a pandemic, even after the WHO declared it one. Wrap it up with a bit of UBI goodwill to soften the blow:

And here they are in full-on conspiracy mode linking the virus somehow to the death of Kobe Bryant and suggesting it also has something to do with Cuomo wanting to be “Big Brother”. It’s actually difficult to follow the layers of disinformation in this one as many of the connections are not even clear. But the message is clear: COVID-19 is to be associated with conspiracy.

Now before you get bogged down in the fine details of these posts, let’s consider what the right thing to do is. The only thing to be saying at a time like this is encouraging people to stay home and wash their fucking hands. These posts are a far cry from the right thing to do.

And this kind of thing was coming up everywhere during the first week or so of the outbreak when people weren’t sure of what to make of it. And I’ve got a particularly rational and scientific facebook feed. I can only imagine what the facebook feed of the person that posted this looks like. They are probably in an echo chamber that resulted in that particular community reacting slower to the seriousness of the threat. Consider this effect nationwide and it means deaths. Potentially lots of them. This is why I’m scared.

What to be done?

Well wouldn’t I love to have an answer. I don’t. But what I’ve been starting to do a bit more is offer (generally) respectful counterpoints when I see blatant disinformation show up in my facebook feed. I’ll leave a comment explaining why it’s not true. Then at least if someone goes through and reads the thread, there’s a bit of exposure to the truth.

Long term, it’s difficult to say. The particular individual I mentioned before is probably too far gone. They have completely rejected science and truth in any of its forms and revel in verifiably false conspiracy theories as a matter of identity and pride.

I guess it will come back to the same thing as always: education. People need to learn how to think critically. Without that, any sufficiently powerful viral thread will kill more people than it needs to.

The path to Machine Learning and AI is not a simple or easy one. On the philosophical side, the field is young and our society is still grappling with what these technologies mean for our humanity. On the practical side, there is no clear one-size-fits-all recipe for an organization that wants to start implementing AI systems. Much to the disappointment of ye old innovation department and impatient executive, we are here to advocate for an approach that is a delicate (and somewhat boring) balancing act between quick wins and infrastructure. Let’s inspect both sides of this AI strategy in detail starting with Quick Wins.

Quick Wins

Few organizations have the resources to dump the millions (or dare I say billions) into true AI moonshots that can take years to produce monetary results. This is essentially VC work and is only possible with piles of cash and an institutional willingness to wait years for an ROI that may never come. Asking organizations to make bets on anything that takes more than some months to see measurable results is not only bound to fail on a project level but actually endangers the future of the organization by disincentivizing them to pursue said path. For better or for worse, most of our economic systems are quarterly based and thus don’t allow long-running initiatives much leeway. This is especially true for the organizations that we aim to help in pursuit of our Vision.

For better or for worse, most of our economic systems are quarterly based and thus don’t allow long-running initiatives much leeway

On a more practical side, Quick Wins can actually help you prepare to play the long game. ML and AI are essentially data driven automations (see this episode of Explained for an excellent non-technical explanation) and anyone who has automated a process knows that you have to understand it first. Furthermore, aiming for Quick Wins will force you to research and build out some very minimal infrastructure which can provide valuable context that is required for longer-term investments. Lastly, and most importantly, Quick Wins provide internal and external stakeholders (especially executives) with the proof they need to justify further investment.

Some real life examples of quick wins that have eventually led to longer-term strategic and infrastructure work:

• 1 week: Data analysis to understand the impact of a price increase on sales.
• 1 week: Analysis of client usage of a web app’s free trial period to understand how much time it takes until the client as effectively converted.
• 2 weeks: Benchmark of a Machine Learning for churn prediction using Microsoft Azure cloud tools.
• 4 weeks: Benchmark of a Machine Learning sales forecasting system to see if a pre-selected minimum accuracy could quickly be achieved.
• 6 weeks: Report generation of clients that are at risk of churn — included charts describing recent client behavior to support decision making.

Give that this blog post has the words “path to AI” , how does it make any sense to include things that are seemingly very basic and generic analysis tasks? It’s simple: working with data is the foundation of Machine Learning and AI. Understanding of your data is essential to be able to train statistical models and analysis is all about understanding your data. It also gives you a chance to use modern tooling which in itself is very informative. If your organization is not able to support a basic analysis task with modern tooling, then it is clear that you have a long way to go. Conversely, if it goes well, the time spent can both generate value as well as illuminate the path forward which gives you your next steps.

working with data is the foundation of Machine Learning and AI

Quick Wins are absolutely essential. Don’t invest millions in long term strategy development. Invest tens of thousands in Quick Wins and watch the long term plays grow from there.

Infrastructure

Making the shift from focusing on Quick Wins to the more strategic focus on Infrastructure doesn’t, on the surface, look that different. You should still maintain an outcome-oriented focus. You should still quickly and continuously deliver value. Key stakeholders shouldn’t be waiting more than weeks for new value to be delivered in key projects. The reason that we start by focusing on Quick Wins has to do with human nature itself and this does not change.

When focusing on infrastructure, the biggest shift is HOW the work is done. Rather than running to the finish line as fast as possible without planning on re-using anything that is built, you widen the size of the effort in order to ensure things are engineered correctly. To understand what I mean, let’s walk a conceptual path using the following stepping stones:

1. Understand what a “Narrow Effort” is
2. Understand what a “Wide Effort” is
3. Understand when and how the “widening” of an effort pays off

Narrow Efforts

A Narrow Effort is, on it’s own, not a bad thing. Narrow Efforts allow for laser-sharp focus on achieving the desired outcome of a single project or initiative with speed as the primary objective. At the very beginning of any project, you should be making a Narrow Effort because you just don’t know if it’s going to pay off or not. You need to reach the desired outcome as quickly as possible just to discover if it can be done! Any extra effort other than what is absolutely required is likely to risk more time, energy, and resources than necessary.

You need to reach the desired outcome as quickly as possible just to discover if it can be done!

Let’s take one of the quick wins that we described in the previous section and take a closer look at it to understand why it is a Narrow Effort. We will use the first one which is “Data analysis to understand the impact of a price increase on Sales”.

• Step 1: Get the data. We email someone asking for a couple of excel spreadsheets for the last 2 years of sales data. They respond in an email with said excel sheet and we put it into a shared file system and it becomes the source of truth for this dataset.
• Step 2: Explore the data. We create a single code repository and start DS Work. We put everything into a few different folders in an attempt to organize things but don’t think too hard about naming conventions, code organization, testing, reviews because all of these take time.
• Step 3: Draw Conclusions. The person leading the project will arrive at a few conclusions. In this case, she conclude that the price increase did not seem to have a significant effect on Sales. She will ask a colleague to review it quickly just to make sure there are no obvious mistakes.
• Step 4: Communicate the results. The Data Scientist that drew the conclusions takes screenshots of the key charts and pastes them into a quickly prepared presentation that can be given to the client. The client understands the results and gives the go-ahead to raise the price of the product across all their stores. We plan to do another analysis in the future to ensure that the conclusions continue to hold.

Note that in this case “DS Work” (Data Science Work) is exploring the data but in general this can also include training ML models, designing dashboards, building ETLs, building data pipelines, etc.

The desired outcome was quickly achieved. However, the technical work and processes were very sloppy and have (at the very least) the following problems:

• The excel sheet was from a person and not the true source of the data which is a database somewhere. It could have been corrupted by human error and you have no way to get an updated version of the data in case you want to repeat the analysis again in the future without involving (potentially expensive) man hours.
• The single code repository which was not tested, organized, and thoroughly reviewed has a much higher chance for errors. Furthermore, if someone else wants to jump in and take the project over in the future, good luck reading and understanding the code that someone wrote as fast as they could over the course of a few days.
• Screenshots into presentations is a very manual process that has lots of places for human error. Plus, the only person that can run the code is a Data Scientist which puts a significant barrier to accessing the knowledge.

Let’s take a moment to visualize a Narrow Effort:

In this Narrow Effort, there is one thread of focus and it is very narrow — the only thing that matters is the x-axis which is time. We are reducing the total number of things that could go wrong and only really trying to do one single thing as fast as possible. Now what happens if we need to do another version of the project in which we need an updated version of the data, make changes to the code, and generate a different type of business value. You’ll get to save some time by reusing some of the code but the project time will still be similar:

Even worse, imagine another scenario in which the original dev is not around and you need to make even the smallest change. The new dev is going to have to read and understand all of the code from top to bottom. They will have to find all hidden assumptions and infer every critical decision because of the lack of documentation. This can result in even the smallest change to what was originally a 2 week project looking something like this:

Anybody who has run a dev team knows what this looks like. And most current data science teams are run like this because they are being run by Data Scientists who have limited experience in the insanely difficult field that is Software Engineering.

As mentioned, a Narrow Effort is not inherently a bad thing. A Narrow Effort does, however, becomes dangerous when confused with a sustainable solution.

Wide Efforts

So imagine that you need a solution to last longer than the few weeks it takes to gain a Quick Win using a Narrow Effort. How can you do this without incurring crippling technical debt? The answer is to widen your effort. As we saw with our Narrow Efforts, there is one single thread of laser-focused drive to meet a very small set of business requirements. So let’s take our previous visualization which only has a single line and add one more:

If we continue with our previous example, adding some cloud infrastructure would now be able to set up a virtual machine that ingests directly from the source database rather than relying on someone to send an excel sheet. You can also store the raw data in a data lake allowing you to have a cheap and scalable way to store all raw data exactly in the form that it comes from. Now, we can share the data with more people and have a way to rebuild the history of our data transfer from the ground up which gives us disaster recovery.

What else can we add?

With tests, we can now make changes to the code with confidence that we won’t be releasing critical bugs that go unnoticed until crunch time. This step must be done with great care because automated tests are written using code which in itself contains the potential for it’s own nasty type of bugs. Now that we have tests, we need to make sure that there’s a repeatable, robust, and predictable runtime environment in which they can be executed. This can be done with CI/CD pipelines which are their own set of technologies.

This is what truly unlocks the holy grail of any kind of software development by making the decision to release a business one rather than a technical one.

So for each subsequent update, you are easily saving days of time. If your project is one that will live for more than a month or two, these savings in time will bring (at the very least) the following benefits:

• Faster time to market
• Smaller technical team required
• Ability to re-train and deploy models faster
• Increased customer confidence
• Happier technical workers

By now you’re hopefully getting the idea. The initial business value that is being generated is the same while you’ve unlocked vast amounts of future business value at a fraction of the cost.

At this point you might be asking yourself “is this still a 14 day project?”. No, no it’s not. And this is where things get tricky and are the difference between an uninformed AI dev team, an inexperienced AI dev team, and an excellent AI dev team.

1. A uninformed AI dev team won’t know how to do it at all.
2. A informed but inexperienced AI dev team will need 4X the time and build technical debt into the infrastructure, tests, and CI/CD.
3. An excellent AI dev team will know the tools, best practices, and have domain experience and can deliver with only an additional 25%-50% of dev time.

Let’s assume an excellent AI dev team and make our final adjustment:

In the case of DareData, a single worker is required to have the skills to do all of this which keeps costs under control. In other organizations this may be split across multiple workers or even departments.

Wide Effort Benefits

Remember how much time it took when we were making a Narrow Effort to make even the smallest change? Remember that, in the worst case scenario of new dev, it could be up to a week? With all of this infrastructure in place to support them, we now look like this:

You can make and release changes so quickly I’m not even able to express it in my little ascii-art diagrams.

Notice that the time to original business value has not changed. The effort has indeed been extended a bit but, much more pronounced, is the significant widening of the effort. Running projects like this in traditional application dev shops isn’t as common as it should be but the best ones definitely do it. Finding teams that work in DS/ML/AI and do this is incredibly rare and this needs to change.

Notice that the time to original business value has not changed

As of this writing, many best practices have yet to be established when it comes to testing and CI/CD for AI systems in particular. At DareData, this is a key focus and one of the main reasons that we are a services company. Although we most certainly do have preferred technologies and methodologies each and every project we work on is, at some level, custom because it needs to be.

For a much more technical and in-depth description, see Martin Fowler’s excellent description of CD4ML. We at DareData proudly implement as many of the CD4ML principles as possible in every project that we do.

The Practical Path to AI

If by now you still remember the original title of this post, you might be a bit confused. What does cloud infrastructure, testing, CI/CD, etc. have to do with AI? Isn’t AI all about Machine Learning algorithms, robots acting like humans, and surprising automations? Well, yes, it is. However a trained machine learning model that sits on a smart person’s laptop doesn’t generate any value. An academic paper showing some percentage increase in accuracy for image recognition does nothing until it is deployed. All of this brainpower and investment is wasted unless you can continuously iterate, experiment, and deploy your solutions.

The current strategy that much of industry seems to be pursuing right now is to hire a bunch of clever Data Scientists and tell them to “do Machine Learning stuff”. 6 months to a year later, they are mired in technical debt. The pile of super smart scientists are spending all of their time debugging code in production, configuring servers, doing basic BI, and complaining that they are not doing Machine Learning. They eventually grow tired of not having a path to success and leave. Sorry not sorry but when working in industry, that’s the job. Even the most research-oriented ML folks working at a lab like Deep Mind spend 50% of their time researching. The other 50% is spent on some kind of engineering.

Can we help you?

If you suspect you have data which is not being used to it’s full potential, please contact me at sam@daredata.engineering. We’re happy to do a free strategic consulting session or two in order to understand if and how we might be able to work together to bring sustainable AI and ML into your organization.

The practical path to AI was originally published in DareData Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

UPDATE: our blog has moved to a https://blog.daredata.engineering/.

The DareData vision is to see machine learning and artificial intelligence democratized worldwide.

As of this writing, world-class machine learning and artificial intelligence capabilities are largely concentrated in the hands of a few huge governmental and corporate players. If not properly distributed throughout the socioeconomic spectrum, it is likely that we end up experiencing extreme inequality. We believe that an effective way to guard against this outcome is to ensure that startups and legacy institutions receive significant support in building out their ML/AI infrastructure and, most importantly, knowledge base.

Who are we?

The co-founders (Nuno Bras and I) are a couple of serial entrepreneurs and technical workers who demand excellence in everything that we do. We have had long careers doing the hard and gritty work of building out projects and companies from the ground up. A sample of companies we have either founded or been deeply involved with from very early on include James Finance, Unbabel, GetSocial, Bond Touch, Wise Metering, Lisbon Data Science Academy, and Daltix.

We’ve also worked within larger organizations leading ML/AI initiatives at corporations.

We have occupied roles ranging from full stack developer to CEO. Our business, management, and technical skills are world-class and we have the track record to prove it.

What do we do?

As the firm Andreessen Horowitz has recently pointed out, working with AI systems is different from traditional software in that it is more manpower-intensive and many best practices have yet to be established. As veteran AI startup founders, we agree 100%.

Despite these challenges, the short-term benefits of AI systems are great and range from internal efficiency gains to competitive advantages over rivals. In the long-term, AI systems will likely be considered necessary for a company’s survival. Reaping the short term benefits as well as making the long term transition needs to happen simultaneously on two fronts:

• Quick wins
• Infrastructure changes

You need quick wins that generate unambiguous value to happen on the scale of weeks or, at the most, months. This shows internal and external stakeholders that continued investment is worth it. You need infrastructure changes because traditional databases, data warehouses, data marts, and code struggle to meet the demands of AI systems and truly data driven organizations. In order to build out long-term and sustainable AI systems, you need to fight the battle on both fronts at the same time and this is what DareData does.

We are an outcome-oriented group that generates quick wins with POCs while, at the same time, building out larger and more difficult strategies and infrastructure changes. We do this with a few different offerings:

• Data strategy development
• Project execution
• Hiring and training

We offer a few cost-free strategy sessions which can be done over lunches or other short meetings. If you like what we have to say then we can move to a first project which is a POC and should be done and delivered within weeks. If a POC is successful and the project continues, you will eventually want your own people working on it which is why we also include hiring and training.

We work hard to de-risk all stakeholders at every step by allowing all contracts to be cancelled at any time. No client has ever taken us up on this.

Who do we want to be?

We are a small boutique consulting company and we intend to stay that way. We are not trying to maximize billable hours. We are not trying to take over the world. Rather, we’re looking to do our small part in creating a world that we want live in by the humble pursuit of our vision.

How can we help you?

If you have data that you suspect you are not using to its full potential, contact me at sam@daredata.engineering and we’ll schedule a short intro call. Or, if you want to move super quickly, just send me a calendar invite and we’ll hope that the scheduling gods are on our side!

DareData Manifesto was originally published in DareData Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.

So no reason to wait! Click here to get your application process started!

Now let’s take a look at some super informative references! So exciting! The calendar can be found here and the wiki can be found here. Keep an eye on them as they may change a bit.

The application process is first-come, first-serve with a single yes/no filter which takes the form of a simple python test. The test is not particularly difficult and it doesn’t need to be because the only requirement is basic python knowledge. We really wanted it this way so as not to tilt the balance in favor of engineers too much (though being a software engineer would help a bit). It’s just because we like interdisciplinary teams!

Seriously, entry-level python is good enough so don’t hesitate to give it a try in case you’re worried about being a good enough programmer, because we want you in the academy!

Step by Step

Application Form

First and foremost: the application form is king! The moment you click “submit” on the very simple application form, you have secured your place in the queue.

Once you finish your application, you will receive a link to the test. Save this link and don’t start the test until you’ve got a few hours to yourself!

Test

From the time applications open, everyone in Wave #1 has 8 days to complete the test. You may start the test whenever you like but you must finish it within 2 hours of when you start it. If you’re an experienced python programmer, this test will not take much time at all but if you’re a beginner, you’ll need the whole time to do the test. That means, don’t start the test until you know you’ve got a few hours of uninterrupted quiet time!

The order in which you finish the test does not matter, only that you complete it within 8 days of applications opening.

Don’t cheat on the test. Seriously, if you do, it will be very obvious when you are not able to contribute any code to your team during the hackathons. Plus if we find out you cheated, you will be expelled from the Academy. Sorry for the bluntness but it kinda has to be this way.

I repeat once more that you are not ranked by your score. You will either pass a threshold that we set or you don’t, and everyone is graded by the exact same criteria.

The test will be delivered using a service called Stepik and you will have to register with the SAME EMAIL as you used on the registration form. Repeat, if you use a different email, you will get lost in the system and not be counted as a submission.

The tests are graded by what your script prints out so be sure to take care with that! A nice hint is to copy-paste the template code every time and leave the input / output sections alone.

Payment

After the tests and applications close, you will have 2 days to make your payment. If you do not make your payment, you will loose your place in the queue and your seat will go to the next person. During these two days you can also opt out of the program and we will give your seat to someone else.

Payments will be done with bank transfer to a Portuguese bank.

Possible Wave #2

We may open a Wave #2 for an additional round of applications. Keep an eye on the mailing list, Facebook, or LinkedIn for information on how the seats are filling. We are expecting quite an influx of applications so be sure to finish the application form and python test ASAP so we can get things settled and focus on getting started with some Data Science!

In Summary

The website has a pretty visualization of the schedule that you just spent 4 minutes reading about ;)

Best of luck!

We had an amazing group in Batch 1 and we can’t wait to see what kind of quality people show up this time around!

P.S. — Study Material

If you don’t know any python and are looking for some resources to get started with, we recommend going through the “learn the basics” section of learnpython.org.

Opening Wave #1 Applications was originally published in Lisbon Data Science Academy on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is a big deal!

Why is this such a big deal

The Lisbon Data Science Starters Academy was bootstrapped with 150 euros and a whole lot of love for our field. We had 15 volunteers working together on a mostly remote basis preparing and delivering course material to a group of students who chose to spend a non-trivial amount of their free time studying and learning about data science. Furthermore, this was extremely accessible! Thanks to our wonderful partners and hackathon hosts we were able to control our costs, as well as expand the network to some of the coolest companies in town.

Think about this for a moment! With a few hundred euros and while continuing to focus on primary commitments such as work, school, and family, you can become an entry level practitioner in one of the most rewarding professions available today! With every new data scientist available in the workforce, opportunities to apply the discipline to a diverse range of areas that sorely need it are opened up. Governance, journalism, biology, chemistry, medicine, social good, gaming… If you have data you can benefit from data science.

So what exactly is the Academy?

It is a hybrid, long-term bootcamp. Students spend the majority of their academy time studying, communicating, and asking questions remotely over Slack while getting together for a hackathon once per month. Specifically, it went like this:

• One 3 day kickoff event that consisted of two intensive training days and one hackathon.

• 5 months, each of which followed a theme and had a hackathon on the last Sunday of each month. The themes were binary classification, time series, recommender systems, NLP, data wrangling, and model deployment.
• The last hackathon was a capstone project in which the students trained a model, wrote a report with EDA findings, and deployed their model behind an HTTP server while a simulator sent observations to their models over the course of 3 weeks.

The graduates are the ones that completed the capstone project and passed a 5 person grading committee. There are currently about 5 students doing round two of the capstone because of scheduling conflicts with the deadlines so we might end up with 12 graduates!

The good, the bad, and the ugly

So a group of 16 working professionals came together to develop and deliver a non-trivial curriculum over the course of 7 months. Here’s the material that we covered in the first two days:

Curriculum 2017

So nothing can go wrong, right? Wrong! There were definitely bugs to accompany all the fun. Let’s do a quick retrospective.

The good (some of it anyway)

• 7 new Data Scientists were born!
• Collaboration! People really helped each other a lot. Anytime something needed to get done or someone had a question, an effort was made.
• We covered a lot of material…
• The hackathons were fun — we got to see some really cool offices of the hosts.
• A lot of people met and worked together that normally wouldn’t have. It’s one thing to hang out at a meetup and it’s a completely different thing working toward something together.

The bad (some of it anyway)

• The 12 hour hackathons schedule was a bit packed. 12 hours for 7 teams to do EDA, train a model, write a report, and give the report was a bit much if preparation hadn’t been flawless.
• The learning material each month was not properly incentivized. They were ungraded and could be consumed passively without having to get your hands dirty.
• Low graduation rate. Depending on who finishes this round of the academy, the highest graduation rate we can expect is 25%.

The Ugly

• Only 4 female participants. Diversity is a big topic among the volunteers and we’ve summoned our network of advisors to help us tackle this issue.

We are working hard on all of these issues and are committed to making improvements in the upcoming Batches. Stay tuned for another blog post about how we tackle these issues and what the results were.

Wrapping things up

Speaking personally, this was an amazing experience. I got to know a lot of professionals both in and out of my field. I hope to get to know many more and be part of a community that is stronger because we spend quality time learning things together.

I now have a whole new group of people to work, learn, and solve problems with and it just feels really good. The future looks bright :-)

What next?

We will be announcing the Batch 2 application opening date within a week. Participants should plan to spend 10 hours per week on average for the Academy. There will be 50 slots open. Applications are accepted on a first come, first serve basis, so subscribe to our mailing list if you want to get notified a day earlier than the official announcement.

In the name of love (for Data Science) was originally published in Lisbon Data Science Academy on Medium, where people are continuing the conversation by highlighting and responding to this story.