I recently joined Zola Electric as a QA Engineer and the Software Development team currently leverages the Flutter SDK to develop their customer-facing Android and iOS apps. The choice of the cross-platform mobile development framework was a no-brainer; Flutter’s hot reload feature, support for beautiful Material Design and Flutter’s ability to incorporate all critical platform differences to provide full native performance are the common reasons for the choice. But then, my fear was how to automate tests for these mobile apps.

First and foremost, testing using the Flutter driver was a bit different from Appium which I was accustomed to. In Flutter, automated testing falls into three categories; Unit testing, Widget testing, and Integration testing. Unit tests and Widget tests focus on individual classes, functions, or Widgets. However, integration testing tests how the performance of an application runs on a real device. This includes testing how the app behaves or responds to a tap, scroll, text input, etc. Integration tests can be automated to run on an emulator.

Secondly, it meant learning a new language; Dart developed by Google. It was not so difficult writing my first few tests because I was familiar with Java. In order to be a bit more comfortable with the language and framework, what I did was:

• create a simple dummy app then set up Flutter driver
• collate all Serialized Finders (objects repository) in a separate file
• write a few tests using this as a guide

Then I ran the test. Imagine my reaction when they all passed!

Another thing was that I was required to share the same repository with the developer. Before now, my experience at my previous companies was that it was better not to co-locate app and automation code as it was assumed that it was too much of a risk to allow testers to have access to the ‘almighty’ app code. My other fear was sharing the same repository with a developer who was no ordinary developer at all! She is Annyce Davis, an Android Google Developer Expert, Software Group Lead at Zola Electric and my line manager.

I was initially afraid that I would not be able to write intelligent code. Turns out, there wasn’t any reason for such fear as it has been a great experience. I am even getting better at it and getting more familiar with the internal workings of the project. Other advantages I have actually experienced are; unreliable web element locators that cause flaky automated tests are avoided and that it also makes each party more involved/carried along with the other’s contribution to the repository. With this experience, there are many other advantages that can be derived from this which can be seen here.

Also, Flutter’s support community isn’t so large yet at least compared to Appium’s. And so I made a few mistakes, figured out some things on my own and asked a lot of questions from the developer who was way more experienced using it and was glad to help.

So far, I’m enjoying working with Flutter. I’ll keep on practising because I have learned that “practice is improvement, and improvement only leads to perfection” ― Richelle E. Goodrich.

My Experience Working on Flutter for Integration Testing was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

How do we maintain the consistency of our GraphQL APIs over time, while the underlying data structures change? This is a common question we face building software on the web. We have mobile and react applications, or third party consumers, that rely upon the structure and data provided our APIs. The truth of our data storage may be much different than the API presents. Let’s adapt a few common scenarios using features available from Absinthe in Elixir/Phoenix/Ecto.

One technique for handling this is to continue presenting the original field, but using data from another source. I would refer to this as “aliasing” or mapping one field’s data into another. The fundamental idea in Absinthe is that the name of the field presented in the API is not required to be the key for the source of data we use.

I’ll cover two scenarios of database refactorings I went through recently, while keeping the API backward compatible:

1. Renaming a column
2. Moving a column into another table

Renaming a Column

This is a simple example to get us warmed up, but we’ll use this technique in combination with another in our next example.

We’ll start with a table and a column (example from Postgres):

CREATE TABLE firmware_versions (
  id SERIAL,
  semantic_version_number text
);

CREATE TABLE devices (
  id SERIAL,
  firmware_version_id integer REFERENCES firmware_versions(id),
  desired_firmware_version_id integer REFERENCES firmware_versions(id)
);

And the Absinthe object type would look something like (I’ve omitted data loaders and resolvers):

@desc "Firmware version"
object :firmware_version do
  @desc "Database ID of the firmware version"
  field(:id, :integer)

@desc "Firmware version expressed as a semantic version number"
  field(:semantic_version_number, :string)
end

@desc "A network-connected device"
object :device do
  …
  @desc "Current firmware version of the device"
  field(:firmware_version, :firmware_version, resolve: dataloader(Device))

@desc “Firmware version we are attempting to update to"
  field(:desired_firmware_version, :firmware_version, resolve: dataloader(Device))
end

Changing the object_type

What happens when we change the column name from desired_firmware_version_id to requested_firmware_version_id? This is an admittedly contrived example, meant to highlight a point. Aside from the normal database migration to alter the table, we need only make a straightforward change in our GraphQL type for the device to maintain the interface for our users.

field(:requested_firmware_version, :firmware_version, resolve: dataloader(Device), name: "desired_firmware_version")

Two changes are required. To start, we update the first argument in field to match the new name of the column (in this case, an association). Then, we add the option for name and pass in a string (binary) to match the previous column’s name in the API.

If you are planning to deprecate the old field name over time, add the deprecate option available on field and pass it a string with the reason for the deprecation. Absinthe will handle the rest.

Moving to a New Table

What happens if we make a more drastic change to the structure of our data by adding a relation to the mix? Whether this is a new table or an existing table that better suits our data, our structures may change in more substantial ways. In this example, we’ll start from the same initial structure of the devices table from above. Instead, we’ll add a new table rather than renaming our column.

CREATE TABLE firmware_update_requests (
  id SERIAL,
  pending boolean DEFAULT true,
  device_id integer REFERENCES devices(id),
  firmware_version_id integer REFERENCES firmware_versions(id)
);

CREATE UNIQUE INDEX pending_request_index ON firmware_update_requests(device_id, pending);

This change will necessitate more changes than our previous example, but fundamentally, the technique is the same.

Add an Ecto Virtual Field and Association

To make it work, we just need to get the data from this new table’s firmware_version_id into a place on the device record where our alias Absinthe field can get to it.

We’ll use a combination of two Ecto features: we’ll make space for the firmware_version_id using a virtual field in the schema and add our association without defining a field (since we already did it in the virtual field).

# in MyApp.Device module
schema :devices do
 # snip …
 field(:requested_firmware_version_id, :integer, virtual: true)
 belongs_to(:requested_firmware_version, MyApp.FirmwareVersion, define_field: false)
end

This creates a place for us to join data into the device, and allows us to look it up as an association.

Update the Absinthe Resolver

Now, we can change our query to load this data. For Absinthe, we do this in the resolver:

defp with_pending_request(query \\ MyApp.Device) do
 from(
   d in query,
   left_join: r in MyApp.FirmwareUpdateRequest,
   on: d.id == r.device_id,
   on: r.pending == true,
   select: %{d | requested_firmware_version_id: r.firmware_version_id}
 )
end

Let’s break this down. We’ll start with the query argument. This is an Ecto.Query for records from MyApp.Device. Next, we left_join our firmware_update_request record on the matching device_id and we restrict it to pending requests so that we only join, at most, a single record for each device.

Lastly, my favorite bit, is the select. It merges the firmware_version_id into the new virtual field we added to the MyApp.Device schema. And with that data in place, we can use our previous change to the Absinthe object type field for the name option and everything will work.

All Together Now

With all of these changes: using the resolver to load the firmware version from the firmware_update_request association into an Ecto virtual field and aliasing the field in Absinthe we have successfully mapped a column from an associated table into place on the original field in our GraphQL API. Thus, we have retained the external structure and data of our API while modifying the underlying data store.

While this is how I solved this particular problem it is, of course, not the only way to solve this problem. Other solutions might involve resolver callbacks (coming in the next version of Absinthe) or simply a custom resolver that returns a properly formed map. Whatever path you choose, I hope this provides you some new info on both Absinthe and Ecto and their function.

Aliasing Fields on Ecto Associations in Absinthe for GraphQL was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

ButterKnife was very useful for us while we had it until it wasn’t. Our gradle build speeds were slowing down and avoiding annotation processors was a small step in the right direction. We also were happening on a lot of errors that were usually resolved with a clean and rebuild, but sometimes required invalidating caches and restarting Android Studio.

There are a lot of articles that already explain what Kotlin Android Extensions are and how to set it up (I recommend this article by Antonio Leiva) so I won’t be covering that here. We used Butterknife in a lot of different ways in our app—to bind values, bind views and set listeners — and I’ll explain how they were replaced in this article.

Binding values (e.g. booleans, dimensions)

Here’s an example of how we used the BindDimen annotation.

@JvmField
@BindDimen(R.dimen.height)
var dialogSizePx: Int = 0

dialog.window?.setLayout(MATCH_PARENT, dialogSizePx)

And here’s how it was replaced:

dialog.window?.setLayout(MATCH_PARENT,
                         context?.dimen(R.dimen.height))

Binding views and setting listeners

In Activities

We now call views in anActivity with their ids. The major things to watch out for are the listeners which were previously called with an annotation on a method but now have to be set manually.

Here’s how we did that with Butterknife:

class MainActivity {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
    }

    @OnClick(R.id.cancel_button)     
    fun handleCancelClick() { 
        // do something      
    }
}

And how we do that now with the Kotlin Android Extensions:

class MainActivity {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        cancel_button.setOnClickListener { handleCancelClick() }
    }

    fun handleCancelClick() { 
        // do something      
    }
}

In Fragments

This works the same way as in an Activity but with one major difference. A NullPointerException occurs whenever a view is called from the onCreateView method. All views should be called from the onViewCreated method instead.

In Custom DialogFragments

Even though the DialogFragment extends the Fragment, the onViewCreated method is never called by default. This is because the onViewCreated method should be called immediately after onCreateView but since the onCreateDialog is used instead in aDialogFragment then that never happens.

So, overriding the onViewCreated method here will not result in any action. Instead, we can override the getView method and pass in a containerView. This will allow us call the views normally from the onCreateDialog method.

val containerView by lazy {
    inflate(context, R.layout.fragment_dialog, null) as ViewGroup
}

override fun getView() = containerView

override fun onCreateDialog(savedInstanceState: Bundle?): Dialog {
    val builder = AlertDialog.Builder(requireContext())

    text_view.setOnFocusChangeListener { v, _ -> closeKeyboard(v) }

    builder.setView(containerView)

    return builder.create()
}

In Custom Views

We implement a LayoutContainer and override the containerView. The layout is inflated in the initializer block and listeners are set in the onFinishInflate method.

class LoginMethodView @JvmOverloads constructor(
        context: Context,
        attrs: AttributeSet? = null,
        defStyleAttr: Int = R.style.AppTheme
) : ScrollView(context, attrs, defStyleAttr), LayoutContainer {

    override val containerView: View?
        get() = this

    init {
        inflate(context, R.layout.view_login_method, this)
    }

    public override fun onFinishInflate() {
        super.onFinishInflate()

        hello_button.setOnClickListener { sayHello() }
    }
}

In ViewHolders

We implement a LayoutContainer in the ViewHolder class, override the containerView in the constructor and handle any listeners with an initializer block.

With ButterKnife:

inner class ViewHolder(v: View) : RecyclerView.ViewHolder(v),    View.OnClickListener {

    @BindView(R.id.appliance_image) lateinit var applianceImage: ImageView

    init {
        ButterKnife.bind(this, v)
        v.setOnClickListener(this)
    }

    override fun onClick(v: View?) {
        // handle view click
    }
}

With KAE:

inner class ViewHolder(override val containerView: View)
    : RecyclerView.ViewHolder(containerView),
      View.OnClickListener,
      LayoutContainer {

init {
        containerView.setOnClickListener(this)
    }

    override fun onClick(v: View?) {
        // handle view click
    }
}

This was a very huge change for us and it took some time to complete, but we believe it’s a step in the right direction. Have you found other ways to use the Kotlin Android Extensions? Please let us know in the comments!

Migrating to Kotlin Android Extensions was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

From previous blog, you probably have a clear understanding of how Data Infrastructure looks like at ZOLA Electric.

In this blog I will describe steps and measures we used to fix ETL errors as well as best practices we have adopted. Before I give you a description on how things used to be and how we handled them, let me give you a complete picture of internal ETL performance at ZOLA Electric for the past one year.

How it used to be
When building a Data Infrastructure, things can easily go out of hand due to bad designs or bad anticipations from the ETL flows that can lead to errors.

Few years ago at ZOLA Electric, the error rate in our ETL infrastructure was very low. This did not indicate that everything was running smoothly but it was due to many errors that happened without a proper way to capture them and provide notifications.

The picture below shows a low error rate before we started alerting on ETL errors.

The Transition Period
To fix problems on “how things used to be”, we had to do a number of things as a way to make sure that we have proper procedures to follow when designing and maintaining our ETL. Our transition period involved changes in following key areas:

Tools and Tech side:
1. Removed the problematic ETL parts (ones that became non-stable).

2. Introduced more advanced and easily maintained tools.

3. Started to frequently Review PR (Pull Requests) from dev codes.

4. Added automated tests that check for code quality and styles.

5. Introduced staging environment for testing all changes before pushing them to production. (This is where we captured most of errors and fix them).

ETL Execution:
1. Introduced explicit job dependencies that stop the entire chain of jobs if an error occurs in one of the dependencies.

2. Allowed jobs to retry in case of connection faults or network related errors.

3. Introduced jobs periodic runs.

Communication:
1. Produced more actionable alerts via e-mails or slack notifications.

2. Team members provide details on what, how and status of what they are fixing from reported errors.

Documentation:
1. Document frequent errors and best ways to avoid them and fix them plus possible ways to automate the process of detecting those errors and fixing them.

2. Redesigned parts that failed too often or had design issues.

3. Documented all parts of our ETL and how to maintain.

After establishing proper steps, things changed in our ETL. Picture below will show how things were during transition period. This is when we introduced proper alerts per each event in our ETL.

High error rate at transition period was mostly due to uncaptured errors on the old ETL infrastructure. The main improvement at this stage was having proper procedures to capture and fix all errors.

Fixing the majority of captured errors at this stage gave us a very good foundation to where we are now and where we want to be in future.

Where we are and where we are going

After introducing standard ways of design and maintaining ETL at ZOLA Electric, our overall ETL error rate dropped significantly.

Current situation: More stable ETL with low error rate.

With current design of our infrastructure, our ETL is more stable and whenever errors occur, they now get captured, fixed and documented. If they are caused by connection / network related faults they even get fixed automatically, with periodic retries. Other errors like bad syntax in code rarely reach production due to code checks we have before deployment and thorough tests on staging.

A portrait of ETL Error Handling was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

At ZOLA Electric, we use the Python Luigi framework and SQL for building our data pipelines that load data to our Data Warehouse currently running on Redshift. Since our data volumes are not in the terabytes range, our existing system is organised mostly around Python that does the initial fetching and parsing of the data as well as the SQL code that is executed against Redshift to structure the replicated tables into report-ready tables.

Production and Staging Environments

We use two Redshift environments, one for production one for staging. Here by staging environment I mean testing environment. So staging environment should not to be confused here with the staging area of a typical Data Warehouse environment. Normally companies do not use a completely separate staging environment for their data warehouse since it is more work to maintain and also expensive for some. Usually, there are special schemas in production instead that are used for testing. We have found that having a separate database instance is very useful to have. It allows us to test new features without breaking the existing functionality. Yes, sometimes we need to sync the Staging but we do it with scripts so it is not so difficult. Other than these basic two configurations mentioned, we also keep local configurations which are used to test newly developed code.

Our Luigi Setup

We run a Luigi Docker container on an EC2 instance. The luigi.cfg.* files are getting copied over from an Amazon S3 bucket upon deploy via Jenkins. We have two separate Luigi config files, for each of the environments, Production and Staging. We have a dedicated EBS volume for keeping scratch files during operation of Luigi tasks.

The Luigi jobs are scheduled via crontab to run on the container. We are also considering running those via Jenkins. The Luigi container itself is running Python 2 but recently we have migrated all our code to run on Python 3 within a virtualenv. Python dependencies are handled via single requirements.txt. More on that migration in a follow-up post.

Luigi Visualiser and Task History

For enabling the Luigi Task History and Visualiser we spawned a separate Postgres instance to keep the history. To enable tracking the visualiser history we had to set the following in ourluigi.cfg.* file:

[core]

error-email:
email-type: plain
email-prefix: Production # or Staging
environment: production # or staging

logging_conf_file: config/logging.conf

[task_history]
enable: True
dbconnection: postgres:///…
pickle-path: /tmp/…

[scheduler]
record_task_history: True
state-path: /tmp/luigi-state.pickle
remove_delay: 86400

Replicating Data from Source Systems

Each data source that we use is replicated to our Redshift in a separate schema. This is convenient for various organisational reasons, at the very least it simplifies the naming of tables and keeps the namespace neat. The replication job is a Luigi Task which runs through a yaml file which usually contains a whitelist of all the tables which we want to replicate along with a method of replication (full or incremental) and then replicates them by using S3 as an intermediate.

Structuring Data into Data Warehouse Tables

Starting from replicated tables, we aim to create fact tables, which store periodic snapshots of our most used entities from our in-house developed operations management system. When needed, the fact tables are reloaded via master tables which keep history. The fact tables are not part of proper star-schemas, as we do not use any dimension tables. Instead, we store the dimension and fact attributes in a single table row of a fact table. In that sense, our fact table layer consists of dimensionless fact tables/flat star-schemas which are then used by Looker. We found that this approach significantly simplifies our data pipeline logic, as well as table development and maintenance. For other sources which do not keep history themselves, we store the history in DWH in separate schemas by doing a periodic pull from them. Overall, we have four different types of tables in our Data Warehouse:

• replicated source data — replicated data from external source systems, which are either RDS or API-based, separated into schemas. Mostly Python code is used for this replication.
• reference — internal lookup tables, used for reporting labels etc.
• extended — calculated data for source tables, usually copies of source data with added calculated columns. Also data preprocessed by Python is loaded here.
• fact — daily snapshots of combinations of source data tables, used as main tables for reporting.

Organising SQL Code

Starting from the replicated tables, we use a combination of Python and SQL to populate the reference, extended and fact tables. Usually there are many SQL scripts involved, all of which have some dependencies among each other, for example one file needs to be executed before some other file, etc. Luigi is great for handling those kinds of dependencies between tasks. Some useful functionalities were added to the plain Redshift SQL scripts via annotations.

The annotations are comments in the top part of the SQL file which are then interpreted with our in-house Luigi tasks prior to executing the SQL code itself. We call this enriched SQL code as well as the framework that executes it FreeSQL due to historic reasons. It is in fact a legacy on top of which we built additional functionality as it proved very useful for defining SQL based transformations.

Most of our fact table preparation jobs are FreeSQL code which have indicated in their header via annotations a list of other FreeSQL files as dependencies that should run prior to the main script. As mentioned we added many other features to FreeSQL but this is out of scope of this post.

Here is an example of a simple FreeSQL script load_table_c.sql:

-- requires:
--  files:
--   load_table_a.sql:
--   load_table_b.sql:

insert into table_c(c1, c2, c3)
 select a.c1, b.c2, b.c3
  from table_a as a
  join table_b as b on a.id = b.id;

In this example, the load_table_a.sql and load_table_b.sql will be executed first as indicated by the comment in the header. After this, the main insert statement will be called. This script will be expanded by our FreeSQL framework into the following Luigi Task DAG, and then executed:

Our Python-based replication and parsing tasks are also organised into DAGs but these are often generated with Python and involving many steps, like for example.

Handling Errors and Alerting

Every time a Luigi task fails, we get an e-mail message and also sometimes for more severe issues, a Slack alert. More details about error handling are coming in the following posts.

Testing Python Code

We use PyTest four running all our Python tests. On each deploy of our Luigi docker container we run these tests. We also have support for testing FreeSQL code, but that is a topic for another post. For test coverage we use pytest-coverage. We are running the complete test suite once a week and then sending the obtained test coverage report via Slack.

Self-service Luigi tasks at ZOLA

Besides the scheduled Luigi jobs, sometimes certain tables in the Data Warehouse need to be refreshed manually by the BI team. For these purposes we developed a simple Jenkins job called Luigi Runner that can be used to trigger Luigi jobs manually.

How this Luigi Runner utility works in detail will also be a topic for a future blog post.

At ZOLA Electric our data pipelines are based on the Python Luigi framework and SQL running against Redshift. In the following blog posts we will dive into more detail, so stay tuned!

Hope you enjoyed this post! Have any questions or thoughts? Let us know in the comments!

Luigi Data Pipelines Setup was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Zola Electric, we were recently looking to hire a Junior Android Developer and this meant we had to conduct interviews for a couple of applicants. Since this was a Junior role, we had no major expectations. We were looking for people who had basic experience with Android development, some source control and testing knowledge, great communication skills and were looking to level up their development skills.

There was an initial phone screen where candidates were shortlisted based on the above criteria and after that, the Android team (which consists only of Annyce Davis and I) decided on a pair programming session. Before this, I had never conducted an interview of this nature but the aim resonated with me. Since we were going to be working closely with this person everyday, it was best to see what a pair programming session with them would feel like as this was going to happen a lot if they were hired. I was a bit nervous before the interviews started and had to remind myself who the interviewer was even though I was the one typing the code. 🙈

Every interview session was conducted remotely via Zoom and started with me sharing my screen with the applicant and explaining what our app did. Then I picked up a ticket from Jira and walked through the description and acceptance criteria with them. I also opened up the designs for the ticket on Zeplin to ensure that we fully understood all the changes we needed to make to the app, down to the dimensions. I did all of this before opening up Android Studio. Because I wanted the whole process to feel less like an interview and more like a typical day at my job, we googled some things and checked the Android Developer documentation together as this is a very common occurrence for me. There was time for some testing before wrapping up the one-hour session.

Just from us attempting to implement the ticket together and me asking them a few questions, I was able to learn a lot about each candidate and was delighted to receive implementation suggestions I didn’t think of myself. Here, I was looking out for a knowledge of Android Studio, basic knowledge of testing, good questions asked and their reasoning ability. Apart from assessing the candidate, my other objective was to ensure that they learnt something in that time frame even if they can’t all be hired. From the feedback I received from all of them, they all did and that’s a win-win on both ends.

I think we were able to pull this off because we were interviewing for a Junior Android Developer role. It will be interesting to see how this can be adapted to more senior positions.

Have you ever interviewed or been interviewed in this way? How did it go? Please let me know in the comments!

Better than the whiteboard? was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

You will fall in love with this role if you can get passionate about bringing solar energy-as-a-service to the off-grid market (over 1.3 billion people around the world) and the unique social, technical, and business challenges that come along with it. Naturally, you’ll be eager for adventure, inspired by the opportunity to unleash whole continents of human potential, and motivated to help build a truly epic business.

As a core member of the Digital Platforms department, you’ll help define and create new product features and experiences for our rapidly growing customer base and workforce. Our team is responsible for Zola Electric’s core platform, our “operating system for the business.” The Zola platform provides much of the magic behind the three key areas of our integrated operation:

• Leading our distributed workforce: automatically dispatching and coordinating the activities of thousands of field staff in multiple countries
• Customer payments and service: enabling all mobile payment transactions and delightful customer interactions for our growing customer base
• Distribution and delivery of solar systems: handling the movement of massive numbers of solar systems and appliances such as low energy TVs from factories through home installation.

Zola Electric is creating opportunities for thousands of local workers to participate in a high tech startup, and our platform enables us to do this with utmost efficiency, scale, and reliability.

The Role

As a Junior Android Developer, you will help drive the evolution of our mobile application as the company accelerates into hypergrowth mode. This includes implementing new features to enhance existing user flows, as well as fixing bugs which hinder successful user journeys. This would include a basic understanding of our APIs and how to work successfully within their constraints.

You’ll be working to solve interesting technical and human problems, such as:

• Implementing an adaptive user interface to accommodate a range of user roles
• Ensuring that our app elegantly handles a range of data network access scenarios
• Collaborating effectively in a distributed team, using a pragmatic, Agile process

Job Requirements

• Initial experience with Android
• Experience with Java or Kotlin
• Initial experience with version control (Git, Github, etc)
• An example project that can be shared considered a bonus, if available
• A Computer Science degree is not required. Experience in internships or alternative career paths are accepted

What Zola Electric Offers

How Zola Electric will set you up for success, we have been growing and developing our team plans, processes, and support structures for years. We are looking to add a small new team together in Nigeria to come onboard and support each other all at the same time.

• Competitive Salary
• Access to the best machines, tools, and internet around
• Paid Conference budget, and paid training materials available to you
• Each hire assigned an experienced buddy/mentor
• Working with existing Technical and Operations Guides on the ground
• Career Guidance and Growth
• Pair Programming Time and Support
• Internal Dev talks, learn from our talks and develop your technical speaking skills
• Travel opportunities to work with our globally distributed team in person

What You Bring

Always be Learning. You are committed to self-improvement, which means you’ve sought and found mentors to learn from. You’re also a natural at collaborative learning including reviewing others’ work, participating in design discussions, improving team code quality, and bring new ideas to the team. Because you know there’s always more to learn, you don’t get hung up on knowing everything before taking action. As you know so well, taking action is often the only way to discover if something works.

Technical beginnings. You have worked with and explored several frameworks and languages. You have started to develop favorites but enjoy learning and digging into new things. You want to dive in on both smaller and newer applications and large applications with years of operational experience under the hood.

User-centered. As much as you love technical challenges for their own sake, you’re inspired by making things that work for real people. You ask tough questions about how requested features solve user problems, and have a great track record of collaborating with designers and product managers on solutions.

Effortless Communicator. You love to work as part of a distributed team. You realize that the key to successful remote work is trust, which comes from regular communication through tools like Slack, and respect for the team’s Agile process (e.g., Jira stories and GitHub pull requests). You also know the value of face-to-face interaction, even if only through web conferencing tools, and have enjoyed being a great teammate in virtualized contexts.

Pragmatic. In a discussion of how to build a feature you will reliably bring people back to the question of what are we trying to achieve, and how will we know when we’re successful? You love new technologies, but will only recommend them if they are the most effective way of solving the problem at hand, and are an expert in playing a devil’s advocate of your own ideas. You are drawn to measure the results of changes to user experiences and look for opportunities to remove features if they are not being used (or used well).

Curious. Unsatisfied with accepted wisdom, you’ll ask why multiple times when wrapping your head around a new concept or proposal. You’re also fascinated with the kinds of problems your peers in the industry are solving, and the novel approaches they’re taking. This leads you to be involved in local user meetings, online communities, and conferences.

Other information

You can learn more about us on Twitter, Facebook, and our Website. If you are interested in joining our team in Lagos, Nigeria, you can submit your resume via our career page.

Help us Power the World with Tech was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

Zola Electric is the leader in providing sustainable, affordable energy solutions for the world’s most complex markets. Today we deliver solar energy to nearly a million people in Sub-Saharan Africa, enabled by innovative hardware and software. Our team is responsible for Zola Electric’s core software platform–the operating system for the business–and the applications and data tools that sit on top of it.

As a senior technologist in the Digital Platforms department, you’ll be instrumental in architecting and developing the systems that enable our rapidly growing international customer base and workforce. You will drive the primary architecture and design of our centralized API and integrations application.

Essential Job Functions

• Guiding a large Rails application towards a more mature modular direction, with an increased focus on ease of modification and configurability
• Helping design a clearer vision of the boundaries and patterns driving the future architecture of the app
• Enabling robust integration support for both in-house and 3rd party integrations, helping us get the most out of integrated solutions and offloading anything not core to the business
• Continuing to migrate from REST APIs towards expanding our platform’s next-generation GraphQL APIs, adding functionality and robustness to our growing internal and partner APIs
• Develop processes to improve code quality for growing team, carrying out code reviews as part of the continuous deployment pipeline
• Participate in Pair Programming on a semi-regular basis assisting developers
• Work closely with Product Management and the dev team developing features that add clear value to our users
• Building workflow-based UIs with React for the independent, web-based static front-end client
• Maintaining and incrementally improving a large Application with 5 years of history, both full of powerful historical insights and not entirely understood corners
• Collaborating effectively in a distributed team, using a pragmatic, agile process

Ideal Qualifications

• 7+ years’ experience with web backends
• 1-year experience with any front-end framework (React, Angular, etc)
• 2+ years experience with Rails web framework
• Experience with TDD, Git, Microservices, and Performance Optimizations
• Experience in production debugging, monitoring, and maintenance
• Experience with Distributed Teams

What Else We’re Looking For

• Lifelong Learner. You are committed to self-improvement, which means you’ve sought and found mentors to learn from. You’re also a natural at mentoring, teaching, and reviewing others’ work, help other engineers make sound architectural decisions, improve their code quality, and get out of their comfort zone. Because you know there’s always more to learn, you don’t get hung up on knowing everything before taking action. As you know so well, taking action is often the only way to discover if something works.
• Technical mastery. You have worked with and deployed several frameworks and languages. You have favorite frameworks, languages, and methods but enjoy learning and digging into new things. You enjoy working on both smaller and newer applications and large applications with years of operational experience under the hood.
• User-centered orientation. As much as you love technical challenges for their own sake, you’re inspired by making things that work for real people. You ask tough questions about how requested features solve user problems, and have a great track record of collaborating with designers and product managers on solutions.
• Clear Communication. You enjoy working as part of a distributed team. You realize that the key to successful remote work is trust, which comes from regular communication through tools like Slack, and respect for the team’s agile process (e.g., Jira stories and GitHub pull requests). You also know the value of face-to-face interaction, even if only through web conferencing tools, and have enjoyed being a great teammate in virtualized contexts.
• Pragmatism. In a discussion of how to build a feature you will reliably bring people back to the question of what are we trying to achieve, and how will we know when we’re successful? You love new technologies, but will only recommend them if they are the most effective way of solving the problem at hand, and are an expert in playing a devil’s advocate of your own ideas. You are drawn to measure the results of changes to user experiences and look for opportunities to remove features if they are not being used (or used well).
• Lateral thinking. In software so many of the leaps forward on the non-obvious ones, hence you have a well-honed sense for knowing when to apply oblique methods to a challenging problem that evades straightforward approaches. This may mean being able to connect the dots across different domains (anthropology, economics, psychology, politics), but it always means being willing to suspend judgment through the process of divergence and convergence.
• Curiosity. Unsatisfied with accepted wisdom, you’ll ask why multiple times when wrapping your head around a new concept or proposal. You’re also fascinated with the kinds of problems your peers in the industry are solving, and the novel approaches they’re taking. This leads you to be involved in local user meetings, online communities, and conferences.

Other information

You can learn more about us on Twitter, Facebook and our Website. If you are interested in joining our team in either the US (East Coast preferred) or Amsterdam, Netherlands, you can submit your resume via our career page.

Help us solve the developing world’s energy needs with software was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

Project delays — in many areas they are so common that they don’t even come as a surprise anymore. Think about the last time you worked with a contractor in your house, the last time you participated in a big business initiative or the last time you heard about another big infrastructure project on the news. Delays seem to be everywhere. On the other side, we spend more and more time and effort on ever-changing project planning frameworks and methodologies, training people over their career in everything from PRINCE II (R) to various shades of Agile. Why is there such a disconnect between what we think we can achieve (and manage) and the actual time and cost of complex things?

Daniel Kahneman and Amos Tversky¹ coined the term “Planning Fallacy” in 1979 and later expanded its definition. Today it describes the tendency to underestimate the time, costs, and risks of future actions while overestimating the benefits. The fallacy can be observed in many settings, from one-person tasks (such as the completion of academic papers² ) to the well-chronicled delays in major construction projects, such as the German Airport BER³.

Kahnemann and Tversky originally explained this fallacy with the planner focusing on the most “optimistic” scenario instead of using their full experience when planning a task or project. While this certainly happens (in my personal experience big companies tend to emphasize “sunny” or “best case” scenarios while planning but forget this emphasis once the predictable delay occurs), even frameworks that specifically account for both known and unknown risks tend to underestimate the total length of a project. Some of the reasons for this are:

• Optimism Bias: The most obvious explanation for underestimating projects is optimism bias. People tend to be optimistic about their personal exposure to risk (i.e. they underestimate their personal risk⁴) and hence tend to be optimistic about their project’s exposure to the same risks.
• Self-Serving Bias: Related to the optimism bias, human beings also tend to experience a self-serving bias in their own past performance by taking credit for things that went well but attributing failures to external factors⁵. This leads people to incorrectly recall how long events took in the past and hence to underestimate future projects. It also, together with the optimism bias, leads planners to underestimate the recurrence of events that have delayed a project in the past, such as vacations, illnesses or the time it takes to plan a meeting.
• Focal Bias: Focal bias describes a bias in which the planner concentrates too much on the part of the problem or project that he knows most about and uses this information to frame the planning decision. As the part the planner knows most about is usually also the safest part of the project (in terms of planning) this leads to underestimation⁶.
• Anchoring and Adjustment: Anchoring and adjustment describes a mental process for judgments under uncertainty, in which an initial “anchor” value is chosen and adjusted based on the information present⁷. Typically, humans under-adjust their initial anchor in this process, hence the results heavily rely on the chosen anchor. This can even be shown if the anchor is obviously irrelevant or nonsensical⁸. In the context of project planning, the reality in many companies is that projects start with so-called “rough estimates”, which are often little more than wishes. However, those wishes have a heavy influence on the final planning outcome by providing an “anchor” for further updates.
• Motivations: Counter-intuitively, a highly-motivated and involved project planner is actually counter-productive for planning success. Individuals who are highly motivated to complete the actual tasks (not only the planning of said tasks) in a quick manner underestimate the total duration of the tasks and hence underestimate total project time⁹ . In business, a similar effect often leads to individuals underestimating tasks that they personally enjoy compared to tasks that they find tedious or boring.
• Perceived Power: When individuals perceive a sense of power, they are especially vulnerable to the planning fallacy¹⁰. The reason for this is the a perception of power tends to narrow the individual’s focus on the principal goals of a project, leading them to disregard peripheral information. This also means that they tend to disregard prior experience and possible obstacles to the project.
• Authorization Imperative: A business-specific reason for incorrect project planning can be found in the “Authorization Imperative”. This describes the reality in most businesses of requiring authorization (often in the form of an approved “business case”) to start and lead projects. This leads to a strong motivation of the planner to report relatively low project costs, which are usually directly and indirectly tied to project length.
• Focus on opportunities vs obstacles: An interesting effect was found between focusing on opportunities and and obstacles of a project. Depending on your mental distance to the project (both in time and involvement) focusing on either opportunities or obstacles can make the planning fallacy worse. This is especially true for projects that are mentally close to the planner (e.g. a project that should be started soon and has been worked on for a while). In this case, if the planner is not actively engaged in visualizing obstacles, he tends to disregard them in the final plan¹¹.
• Asymmetry and Scaling: First popularized by Taleb, asymmetry and scaling refer to the principle that random events are not equally distributed in either impact nor frequency and that negative consequence scale with the complexity of the system¹². Therefore, it is easy to underestimate the negative consequence of random events on a project. This is especially true for complex projects, where positive events usually have a relatively minor impact while the impact of negative random events can essentially be “unlimited”.

Knowing all of these imperfections of the human mind does not necessarily help us improving our project planning capability. However, there are some strategies that can help generating better plans in the future:

• Segmentation: Segmentation describes the breaking down of larger tasks into smaller, more manageable pieces. Those pieces are then estimated separately to come up with a total plan. The principle behind Segmentation is that reviewing each piece individually will give more depth to the planning picture and will usually lead to the sum of smaller tasks being more than the estimate on one bigger task¹³. Many modern software packages use segmentation techniques to improve planning and execution of projects.
• Reference Class Forecasting: A completely different approach to classical project planning is called “Reference Class Forecasting”. This methodology ignores the classic principles of Project Planning such as identifying dependencies and planning discrete tasks and rather identifies “reference projects” that are similar in size and scope and attempts to establish a planning horizon based on those real-life examples. This methodology has been successfully implemented in large government-sized projects¹⁴. Reference Class Forecasting is especially useful to find the correct anchor (see anchoring) for a plan before going too far into detail.
• Impartial Planning: Given that many of the biases describes above especially apply to individuals that also execute the tasks that were planned, it can be useful to have impartial planners who will not participate in the project itself plan the project or at least parts of it. The classic Project Manager which is not responsible for any part of the project delivery but only its organization is born from this idea. This helps especially if a project has many inter-dependencies and moving parts, as an impartial planner does not have a personal preference (e.g. being especially efficient in his or her own tasks) and will prioritize the overall project timeline above all else.

An example that combines these approaches into one was an ERP roll-out I was participating in earlier in my career. In this particular case, the total project consisted of many individual implementations in several locations within one company, following a common business process template and using the same solution.

We were able to increase predictability and precision in planning by using four concrete improvements:

1. First, we obtained rough estimates for a single implementation (one location) by talking to the vendor and external companies who had used this software to give us a baseline (reference-class forecasting)
2. Then, we used a resource who was experienced in projects of this scope without knowing the specifics of the product to build an overall, high-level roll-out plan (Impartial planning)
3. Third, we split up the high-level planning into small pieces to make each step of the way concrete and understandable (segmentation)
4. And lastly, we used the learnings from each individual implementation to improve the plan for the following location in the same way

While our plan still was not perfect, it allowed us to forecast Go-Live dates and a roll-out schedule to a much higher degree than we thought possible at the beginning of the roll-out (we hit all our targets for the first wave of implementations!). Project planning might still be more art than science, but by following what research has shown to work, you can definitely improve your plan!

Thanks for reading and hopefully the ideas above can help to improve the next project plan or, at least, understand the risks hidden in the neat slides and GANTT charts we are all used to.

References

1: Kahneman, Tversky 1979
2: Buehler, Griffin, Ross, 1994
3: https://www.telegraph.co.uk/travel/news/berlin-new-airport-delayed-again/
4: O’Sullivan,2015
5: Pezzo, Litman et al, 2016
6: Buehler and Griffin, 2003
7: Kahneman and Tversky, 1974
8: Strack and Mussweiler, 1997
9: Buehler, Griffin and MacDonald, 1997
10: Weick and Guinote, 2010
11: Preetz, Buehler, Wilson, 2010
12: Taleb, 2007
13: Forsynth, 2008
14: Flyvbjerg, 2006

Why are so many projects late — The science of project management was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.

Drive

• author: Daniel Pink
• theme: Behavioral Psychology
• recommended by: Annyce Davis

This book is all about understanding what truly motivates us as humans and how that knowledge can impact the way we approach work and business.

Remote: Office Not Required

• authors: Jason Fried & David Heinemeier Hansson
• theme: Distributed Work
• recommended by: Moyinoluwa Adeyemi

A great book about how to function as a distributed team. I recommend this to most folks who end up working with a full or partial distributed team. You can read Moyin’s full review in a previous post.

Deep Work: Rules for Focused Success in a Distracted World

• author: Cal Newport
• theme: Time Management
• recommended by: Iris van de Kieft

Deep work is the ability to focus without distraction on a cognitively demanding task. It’s a skill that allows you to quickly master complicated information and produce better results in less time. Deep work will make you better at what you do and provide the sense of true fulfillment that comes from craftsmanship.

Antifragile

• author: Nassim Taleb
• theme: Systems Design
• recommended by: Thor Muller

Taleb challenges how people think about risk, perception and human behavior. Antifragile is possibly his most actionable book for product designers and developers, in that he provides guidance for what any system needs to do if it is to not just withstand shocks and failures, but actually improve when they occur.

The Field Guide to Human-Centered Design

• author: IDEO.org
• theme: Human Centered Design
• recommended by: Nick Stevens

A step-by-step guide that will get you solving problems like a designer.

The Mythical Man-Month: Essays on Software Engineering

• author: Frederick P. Brooks Jr.
• theme: Software Project Management
• recommended by: Dan Mayer

Assigned to me in College, one of the tech books I still think about most often. With a blend of software engineering facts and thought-provoking opinions, Fred Brooks offers insight for anyone managing complex projects.

Looking for More?

Below I will list the rest of the books on our team’s list without all the details.

• Radical Candor: Be a Kick-Ass Boss Without Losing Your Humanity
• Emotional Intelligence 2.0, thx Dian Rosanti
• Creativity, Inc
• The Five Dysfunctions of a Team: A Leadership Fable
• The Advantage, Enhanced Edition: Why Organizational Health Trumps Everything Else In Business
• The Checklist Manifesto, thx Galt Elijah MacDermot
• The Manager’s Path: A Guide for Tech Leaders Navigating Growth and Change
• High Output Management
• The Effective Executive: The Definitive Guide to Getting the Right Things Done
• Managing Humans: Biting and Humorous Tales of a Software Engineering Manager
• Peopleware: Productive Projects and Teams
• The First 90 Days, Updated and Expanded: Proven Strategies for Getting Up to Speed Faster and Smarter
• The Hard Thing About Hard Things
• Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency
• The Righteous Mind
• Thinking, Fast and Slow
• The Art of Thinking Clearly
• Manage Your Day-to-Day: Build Your Routine, Find Your Focus, and Sharpen Your Creative Mind
• Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days

If you want an even more tech management recommendations, I found this post, the Best books for new, first-time managers, to be inspiring as well.

Off Grid Electric Tech Team Booklist was originally published in Power the People on Medium, where people are continuing the conversation by highlighting and responding to this story.