Let me walk you through how I setup things. And hopefully I will figure out the questions I should be asking myself to improve it.

The first step when I got my new Mac was to make sure everything was up to date. So I let my computer do it’s thing and update properly. After that, I reviewed all my trackpad and mouse settings:

• Enable tapping to click, and right-click in the bottom right
• Using the old school scroll direction
• Review all the gestures, and disable Launchpad

Next, I setup my Desktop and dock, and all the hot-corners:

• Top left — show desktop
• Top right — quick note
• Bottom right — turn off screen (which i setup to lock after a few seconds)

I also change my Finder too: to show extensions, delete bin after 30 days. I create a folder called “Developer” where I keep all the repositories I use to work and for personal projects. And I also set my finder to show files and folders as columns.

With these basic steps done, I sign in on iCloud and I install 1Password. I need to start with 1Password to sign into all the apps and websites. After it, I do Obsidian, Xcode and its command-line tools, Tailscale, Slack, Notion, Linear, VSCode, and iTerm2.

I quickly open most apps, just to make sure I do all the MacOS permissions and syncs. For example:

• Reminders
• Calendars, and I setup my Google Accounts (work and personal)
• Maps
• Mail

Finally, I performed all my terminal setup tasks. This means installing Homebrew, OhMyZsh, PowerLevel10k, Python (with pyenv and pipenv), and setup Git.

By setup Git, I mean ensuring that SSH and GPG are correctly configured, so that I use them for my signed commits, and pushes and pulls.

What are my questions after this setup?

• I know Macs have a Migration Assitant — should I be using it and just get data from the old mac? Feels like setting up a clean mac prevents me from bringing clutter.
• How many GPG keys should I have? — The answer seems unclear as we can have either one key and multiple subkeys, or just have multiple keys.
• I wish that I could just install all my normal apps with a single app or system — what is the best way to do it? I know I forgot some apps, and I will only install them when I face the issue they solve.
• I had some issues signing up to google with my security key. It was really weird, and I had to set them again as Passkeys instead of 2FA. I have no idea why they were not working, and I am lucky I did not get locked out. What the heck happened?

Originally published at https://joseantunes.tech.

I said to myself that I was going to write once a month during 2024. I want to make good on that promise, though I am already late for January… So let’s just say I want to write on average once a month. Just a small change of the goal posts. I think it is acceptable.

Goal posts relate to the topic I want to talk about today: management. “Learning how to be managed” is a topic I am interested in, because I think I have been doing it wrong. And it took failing at one of my own goals to dive into it.

My goal, the one I did not achieve, was to get my promotion to Senior Data Engineer. That was my internal 2023 goal. It hurts to say it out loud and admit in public failing to achieve a goal, but self-awareness is the start of change.

The aim of this post is: to write down my understanding of basic management concepts, and to share them with whoever wants to read them. To provide some behavioural tips, actions and a framework to get more out of your meetings with your manager.

My (small) research came from two authors, Camille Fournier and Julia Evans. Specifically, I focused on:

• The Manager’s Path — Chapter 1 — Management 101
• Help! I have a Manager
• Writing a brag document

I will focus on two parts: what a good manager is, and techniques to improve your communication with them.

And finally, before we start, a thank you to all the managers that cared about me! The managers that made sense out of my streams of thought, that were kind, and helped me grow in this career that I enjoy so much!

What is a good manager

I have to start by sharing a simple, yet fundamental, insight from The Manager’s Path:

Some people think a good manager is one that will perform benign neglect, i.e. if you know what to work on, they leave you alone.

That is not what a good manager is. That is also not what a bad manager is. That is just a meh manager ( …a mehnager…).

The experience of having a good manager will define parts of your career, and it will also help you develop your own philosophy of management. It can be an extremely rewarding experience to be a part of.

Having that said, good managers:

• They care about you as a person
• They teach you important skills
• They help you figure out what you need to learn
• They want you to take their job
• They help you understand what to focus on

Good managers will take the moments they have with you ( the 1:1s) and take them to the next level:

• The meetings will be regular, predictable, and private
• They will create human connections, try to understand you as a person and let you understand them
• They will provide you with actionable feedback — even in situations where getting feedback is uncomfortable
• They can help you find resources for growth!

This is probably the ideal manager. The truth is that not everyone can cover all these marks. And that is ok! So what should you do in this situation, when your manager lacks some skills?

The answer is simple, continue networking within your company (and even outside) to find people that are inspiring in the areas you need guidance and your manager lacks. Maybe you need someone more empathetic, or who is a better engineer to discuss and teach you. Whatever that is, those skills should be present around you, and connecting with them will help you be better!

For the sake of completion, let’s also share a few examples of bad managers, again getting information from The Manager’s Path:

• They are neglectful when you need help, and brush concerns aside
• They will avoid meeting with you, or constantly move meetings around
• They don’t give you feedback that is actionable
• They only tell you are not fit for promotion
• They are micromanagers, and/or question every detail of everything
• They don’t let you make decisions on your own

These behaviours will affect your mental health. They make the work experience unpleasant, and chip away at your confidence and willingness to wake up everyday to do things. I think it is worth remembering that people don’t quit their jobs, they quit their managers — or so goes the saying.

Now that you know the difference between a good and a bad manager, let’s talk about communication and how to improve the working relationship.

Communicating with your manager

This is going to sound ridiculous but I have to say it: managers are human beings. All human beings deal with many things in their lives:

• bad nights of sleep;
• families;
• their own goals and aspirations;
• their fears and worries;
• the work and other stakeholders;

So before anything else we should remind ourselves to have empathy when dealing with managers, as they are not at our full disposal. Being a manager is hard.

Having that said, there are a few things we can do to improve our communication with them.

Lets start by talking about 1:1s. These are the meetings where you and your manager align personal and company goals, and discuss projects and priorities.

The first step to having productive 1:1s is understanding they are important, and like all important things in life you should be opinionated about them. I have certainly failed here before by assuming it was my manager’s job to guide these meetings. What does it mean to be opinionated in a 1:1?

First of all, save time to prepare — maybe a 30 min block the day before, so that you can think and still change things before the real meeting. That’s how I do mine. Additionally, you should be able to discuss:

• What has been going well
• What is not going well
• Team priorities
• Asking for opportunities
• Career Planning
• Asking for Feedback
• Giving Feedback
• Do brainstorming sessions
• Ask for resources

Personally, in my new 1:1 document I wrote a header for inspiration (and also to align with my manager) that has the following ( and feel free to copy this):

- **Information**
 - What is (is not) going well
 - Talk about problems early, so that you can work on them
    - If things are not going well, be clear about why and
      what an ideal solution would be
 - Team priorities
    - Knowing priorities will help you be proactive, and come
      up with ideas and organise work better
 - Set clear expectations and goals

- **Help** 
 - Brainstorming solutions
 - Asking for resources

- **Growth**
 - Receiving/Giving Feedback
 - Asking for opportunities
 - Career Planning

I make the above section a collapsible section, so that I can use it as reference but doesn’t become cumbersome. If you think about these sections, you will no doubt have something to say.

Secondly, one important note on setting clear expectations and goals — ensure the are S.M.A.R.T. (I am defo stealing this from one of my friends that is an ace at setting goals).

• S — Specific
• M — Measurable
• A — Achievable
• R — Relevant
• T — Time-Bound

There are many resources to learn how to measure your smart goals — or if you want to be really trendy you can even ask ChatGPT.

And finally, I want to talk about brag documents! These have helped me immensely track what I do (even the small things) and be proud of my achievements. They are especially good as neither you nor your manager will remember everything you did every quarter (or however long your evaluation period is)! Does the question “What did I do these last 6 months?” ring a bell?

There is a lot of good information in the linked article — I will paste what I think are the most relevant headlines and spend time only on a few points:

* share your brag document with your manager
   * They will be thankful that you are helping them remember
     all the things that you have done

* share it with your peer reviewers
   * The people that work directly with you might remember things
     you did for them that they want you to highlight! Remember
     that we work in teams, and the effects that we have on others
     are very important for great performance!

* explain the big picture
   * Why did you choose the projects you did? It is good to explain
     your focus, and ensure that you are taking your career in the
     correct direction

* use your brag document to notice patterns
   * What work makes you feel good, or proud? Are there themes to the
     projects you love being involved in? Answers to these questions will
     help you steer your career in the direction of your interests, and
     overall work happiness

* don’t forget to include the fuzzy work
   * What to say - random tasks that improve quality of life are work too!
     Not to be forgotten, "1% better every day yields a LOT of improvement
     at the end of the year" - https://jamesclear.com/marginal-gains

* encourage each other to celebrate accomplishment
   * Let me quote Julia Evans directly: "It can be hard to see the value
     of your work sometimes, especially when you’re working on something
     hard, and an outside perspective from a friend or colleague can really
     help you see why what you’re doing is important."

I spend a little bit of time every week keeping my document updated. In the beginning it was hard to do, and even harder to share a brag document with my manager. However, the very positive reaction encouraged me to continue. Now I have a list that is starting to make me proud! Here are the sections I use:

* Big Picture
    * defining what my work overall is
* Random untracked bits
    * all the tiny pieces of work that don't really have a ticket
* Area #1
    * My main area of work
* Project Work
    * Projects that I have participated in regardless of the format
      I explain what my contributions were
* Written documentation
  * documents I start, or things I keep updated

These areas work for me, and you have to find the areas that work for you!

This is all I have to say about what I have learned. I hope you find some of this useful in your own career. Go and explore the articles I have mentioned, and the many other available sources online. As I said before, self-awareness is the first step to getting better at something!

If you have any questions, feel free to reach me at “me[at]joseantunes[dot]tech”.

Have a super day,

J :)

Originally published at https://joseantunes.tech.

Pre-requisite: Don’t forget to read the section “Accessing CloudKit Using a Server-to-Server Key” in this page here. You will need to create a key that you can work with.

TL;DR: This GitHub repository has the code I wrote to upload a CSV to CloudKit Database. Scroll to the bottom for a quick heads-up on a hard to find (un)documented behaviour.

Try the code by forking it and first adjusting the Location struct to your needs and then running:

$ swift build
$ swift run csvToCloudKit \
 --ck-key-id "your/key/id" \
 --private-key-file-path "path/to/file.pem" \
 --csv-file-path "path/to/file.csv"

Ideally, a blog post has something new to say. I am not sure my post fits that category to everyone, but it does to me.

A while ago, I had an idea for an app, and I decided to learn SwiftUI and Swift to make it happen. Why those technologies? Because I am a Mac and iPhone user, and I wanted to take full advantage of the libraries available. I wanted to be able to integrate with any possible Apple services.

I have not gotten far in my app, but I did solve one of my first hurdles. As such, I want to share some of the things that I have learned, and the code that I have built.

Let me set the scene: I want to learn how to use CloudKit DB, and I want to seed it with data that will be viewable in my app.

In my case, the seed data consists of a set of locations that are relevant to the users. Though seeding databases is such a common problem, I did not find a trivial solution to seed my database, say: using a CSV and importing it into the table (or set of records) that match the CSV’s schema.

I know that spreadsheets and CSVs are the devil — I have talked about that at previous jobs many times — however, for this quick script, it should be fine. I am only using it for 28 locations, and by the time I need to manage more, I will have a better solution. This is just a proof of concept and a learning experience.

Back to our problem, some of the solutions I saw involved adding the CSV as a part of the app Bundle, and then creating a button that would push the data to the database from my iOS app. This way, it is possible to use the CloudKit library, and a lot of the functionality built into the environment that is used to build iOS apps.

I don’t know if this is the right way, but it doesn’t feel like it. So I decided I wanted to create a Swift CLI script that would take a file and parse each row to create it as an record in the database. Nevertheless, what sounds like an easy task is not. We can’t use CloudKit, we need to use CloudKit Web Services.

I divided my code in three parts:

• The reader that loads and parses my CSV — here
• The models that represent my record and the CloudKit requests — here
• The requests that I need to use to create my records — here

The biggest hurdle was that the type values in the Record Field Dictionary must be upper case! The documentation reads:

A record field dictionary represents the value of a field and is of the form: { value: CKValue, type: string (optional) }. CKValue represents field values of type String, Number, Boolean, Reference, Asset, and Location.

But nowhere does it say that type needs to exactly match the text schema of the database, such as:

primitive-type:
    | ASSET
    | BYTES
    | DOUBLE
    | [ ENCRYPTED ] { BYTES | STRING | DOUBLE | INT64 | LOCATION | TIMESTAMP }
    | INT64
    | LOCATION
    | NUMBER [ PREFERRED AS { INT64 | DOUBLE } ]
    | REFERENCE
    | STRING
    | TIMESTAMPte

Once I tried with upper case values and the code worked, I was super happy!

That’s it! Shoot me any questions if you have them! :)

Originally published at https://joseantunes.tech.

One of the problems I noticed is that contractors were just making the servers running the app bigger to solve issues. That is not the ideal solution — at least when money is limited. So, as part of my job after fixing the initial performance issue, I decided to focus on minimizing the server costs. That implied downsizing the RDS instance running the app.

When you have an app running in production, you want to perform a change without downtime. How can one do that when it comes to the database that powers the application? Whatever we do, how can we make sure we are safe making these changes — not losing data, not annoying our customers?

I spoke with a friend of mine — one of my Software Development gurus — to come up with a plan to do this safely. I broke it down into two parts:

1. Should I downsize my RDS instance? Can the new size handle the current application load?
2. How do I do that with minimal downtime?

To tackle the first problem, we broke this problem down into several steps. To not influence the production database and increase developer confidence, we need to setup a second database, load it with a snapshot from the first database, run the expensive queries against it, and evaluate if we can downsize the DB.

Setting up the database is trivial— follow any RDS tutorial, and you’ll get it up and running in minutes. The second step is to extract data from your current DB. I used the tool mysqldump by running the following command:

mysqldump -h <my-database-host> -u <my-user> -p --single-transaction --quick --set-gtid-purged=OFF <my-db> > prod_db_dump.sql

The command will give you a snapshot of your data without locking your database, and depending on how much data there is it might take a while. It is important to note that because there is not DB lock it might not preserve data consistency i.e. some foreign keys might not have matches, and other issues. When you load the data into your new DB you might come across some permissions issues. This is because the SUPER role is reserved for rdsadmin account, which a regular AWS user does not have access to. Have no fear, just follow this StackOverflow answer.

After setting up the DB I used a tool called Locust — a python based load testing tool — to run queries against it, and see how it performed to a load of the same volume that the production server was receiving.

Feel free to use the code below — but don’t forget you copied it from the internet.

Being satisfied with the results I proceeded to the second part of the plan — downsizing my instance with minimal downtime. I did a lot of reading around how to do it safely, and the agreed method seemed to be:

1. Turn on multi-AZ for your instance if it is single-AZ — no downtime here
2. Scale down your instance — downtime of around 60s while the DNS resolves to the second replica, make sure your application does not fail while the DNS is switching
3. Turn off multi-AZ if you wish to go back to single-AZ — no downtime here

Whereas 60s does not seem like a lot, I was still curious to know more. I ran the load test against the test database with a simple query to see my downtime.

Downtime was a bit more than I expected, but was still manageable. During a time period of low usage I was safe to put this plan into action. Had it been longer or usage higher, I’d need to come up with a different plan.

With all this testing, the plan worked without flaw. In a way, I was still a bit nervous, but I was way more confident after my initial results.

Let me finish with a similar message as my last post: plan, and do your tests. Increase your confidence by breaking non-production services, so that you can change production without fear.

Let me know if you have any questions, and have a great day!

Being furloughed during COVID-19 was tough at times, but in retrospect, it couldn’t have been better for me. I am lucky enough to be in a position where I can bet on myself, and share the knowledge that I have.

A couple of weeks back, a friend reached out to me and asked for help with her website. A really slow website. Considering I was on furlough leave I found the time to take on this puzzle to see if I could help and to learn something different besides the work I’ve been doing on my personal project.

Let’s establish the lay of the land for this investigation.

My friend had a great idea for a website to match people based on relevant individual features. She has all the business know-how but not enough technical knowledge. So, she did what people usually do, she found something ready to use and added some customization to set her platform apart from others. The customization was undertaken by contractors that were present along the way, and made the tech decisions for the business.

The platform grew naturally and organically, and then the users started experiencing some slowdowns. New changes were added to the platform, and things started breaking without particular reason. Luckily, the contractors fixed the bugs they had introduced for free, and as they had full control of the code things were stable for a while. Well, more like things were slowly degrading but appeared stable.

A successful marketing push added a bunch of new users to the platform, and the number of concurrent users saw an increase. This was the tipping point. The platform slowed to a crawl, and became unusable during peak times. In part, before I was brought in, this issue was solved when the contractors increased the size of the machines running the server and the DB — they were now 8vCPU and 32Gb RAM machines — but the platform was still operating sub-optimally.

This was the point at which I was asked to help.

I started by having a couple of long chats with my friend to understand exactly what the story had been up to the point when I was contacted. With that context I would be much better armed to do my job. The initial conversation yielded two clues: the desktop website was slow in general when you went through it, and the mobile app struggled with speed during some parts of the day.

This process of having multiple conversations to try and fully understand the product eventually led me to find out that the mobile app and the website were built by different contractors. In turn, I thought that there was a problem with the DB that needed to be fixed. Knowing that the website was slow in general, the issues affecting it could potentially be related to caching, the size of the instance or even its location.

I requested access to the AWS account that was hosting everything, and while waiting for access to be granted, I inspected the website and spoke to some mates to validate my initial intuition. Their years of experience added a lot of depth to my initial intuition. They certified that I was going on the right track, and the website inspection showed that there was no caching at all (all static assets and images were being served from the server) and that the website itself was sending a lot of requests that weren’t used, along with requests that were taking ages to respond.

Once I did get AWS access, I confirmed my assumptions. There was one EC2 instance running a production and test website, and one RDS instance with MySQL. I got access to the EC2 instance by being given its SSH keys, and saw for the first time the PHP app that powered the website. From the config files I was able to get the DB credentials, and got that access too. Currently, not the safest setup.

The EC2 instance had its own public IP address (where the domain points to) and it also stored all the user files (images, etc, were not in S3). There was an ELB but no traffic was going through it.

I explored the code base to the best of my ability, and tried to understand how it worked. I found the obvious files that had the configs, and some variable definitions.

The other developers were also copy & pasting files to “deploy” to production.

At this point I thought that working with the code to try and make improvements was too risky. It could bring the whole website down.

My first course of action was to bring up a new development server, with an exact copy of the data that was in the production server, so that I could explore it freely without causing any kind of nuclear meltdown on the website. Being confident in breaking things securely allows a developer to experiment as much as they want. That is great!

Once this server was setup, I collected the code and moved it to my machine so I could explore it more thoroughly, and understand my next steps. I didn’t feel particularly confident in changing the code at this stage given that some contractors were still working on it (and PHP is not my favorite language…), so I focused on caching the static assets via Cloudfront.

It was simple enough to find how to only cache certain paths and assets. There are plenty of articles out there explaining how to add free SSL certificates, Cloudfront, ELBs, and connect EC2 instances. What was hard was making sure that the DNS records were pointing to the correct server, and that everything was wired up correctly.

One of the issues that I was getting was a very strange redirect loop, that I couldn’t figure out the source of. I trawled the Apache Server config, and the .htaccessfiles, but couldn’t find anything that would explain this unexpected behavior. Then I went through the settings on AWS to fully understand how each of the components was communicating with one another. Cloudfront was redirecting everything to HTTPS, then communicating with everything in the backend via HTTP. Turns out that the PHP application was redirecting every HTTP request to HTTPS, therefore creating an endless loop.

The only variable that I could tweak in the config was the HOME_URL , and if it was set to HTTPS it would redirect all requests under the hood. This is also the same variable that populates the HTML with the assets’ URLs — so it was breaking the website and delivering mixed content. Once I commented out the PHP code that did the redirect, the cache worked flawlessly on the dev server. Good job, but not enough. The dev website wasn’t even loading when there was peak traffic on the production one, and they shared the same DB. Therefore fixing the database was paramount to bringing stability to the platform.

Performance insights on AWS revealed which were the queries that were slowing everything down. The first one clearly tied with the matchmaking algorithm, and a second one that was tied with updating a user’s timestamp for last time active. The combination of these two was the key to the slowdown that happened when the platform grew.

The query that updates the timestamp is managed by a cron job that runs on the server every minute. It makes sure that users are online or offline, depending on their actions. The updates are set to LOW PRIORITY SQL commands.

The matchmaking query relies on getting the most recently active users, and applying a lot of conditional filtering to find the best matches and calculate a compatibility score. For a start, the conditionals were highly inefficient, and secondly they were running over the entire table of users, sorted by last time active.

When the user base grew, so did this table, and the query became dramatically slower, locking the table and preventing all other updates from happening.

The resulting load made the website unusable for the users — taking 6s or more to respond — and given that the platform relies heavily on the database, even the dev site was broken.

After perusing the code once more, I spotted a config variable USERS_LIMITtied with the SQL queries, and checked crontab -l to maybe change the frequency of the cron job. I decided to open a support ticket with the account manager of the platform, to really understand the implications of changing these values.

In the end, even though I was told that not much would change (strange…), I decided to reduce the USERS_LIMIT variable to a more manageable value. This would reduce the number of users scanned in the table when the matching algorithm was running.

Lo and behold, this change had an immense impact. The load on the DB went down massively, and the website went back to normal response times. There is still a lot to do and optimize, but we can focus on slowly growing again, and now more than ever we have a better understanding of the platform that powers my friend’s idea.

A lesson that we can take from this voyage is that it is important to have someone technical in a team that understands the business needs and constraints. By talking we can bridge the gap between each stakeholder and end up with a product that delivers the vision in a better way. We didn’t need to “build a caching system” like the contractors proposed initially, we just needed to understand our platform, and the business constraints to see that the fix I did was possible.

During the next few months, my job will be to continue making small improvements while also planning out the future of the platform. At the same time I will be assisting with technical communication, and where we should focus our attention next. I want to increase the technical knowledge in this team, and elevate the platform to reach higher goals.

Thanks for reading. :)

Extra minutes for holidays

Imagine yourself in your shiny autonomous car. It is cruising around the south of France, away from all the chaos of the city. Around are all the beautiful hills and mountains, with their light summery green colors.

While your car drives itself, you, your family peer out of the open windows. The landscape keeping you forever in awe, the warm wind blowing your hair back. This holiday was well deserved, the break is blissful.

The car pulls over into a little nook on the side of the road, between the cliff and the mountain. It seems to be the perfect place to stop, breathe, and observe the vast landscape sprawling below you. Everyone in the family comes out of the car, as the engine automatically shuts down.

In the warm summer afternoon, you can’t even tell how much time went by looking at the little villages and valleys afar. When everyone is ready, they make their way into the car, and as you try to set a new destination, it refuses to accept the given coordinates. An error message pops up:

No more minutes available to drive abroad. Contact provider.

This company has benefits

Roberto’s life got easier one day. The company he worked for decided it was good for employees, all employees, to have company approved self-driving cars. During the announcement he couldn’t believe it. Not at all. How could they be giving cars to every single employee?

Roberto had just joined to company as a low skilled machinist, one among many. He worked in shifts, sometimes during the day, other times at night. He lived far away, as rent close to the factory was expensive. Life was hard. Long hours and long commutes meant little rest. Some of his colleagues had even had accidents whilst driving to or from work. Well, not anymore. No one would have to worry again.

Once that policy was put into place the car arrived at his driveway within a week. No driver at the wheel, just the soft electric buzz of the motor announcing its arrival. It was gorgeous. The silver color shining in the sun, caught Roberto’s family’s eye, and all were immediately drawn to it. The kids obviously went in first, as he stayed with his partner outside, basking in the joy of having a new vehicle in the family.

The first week was strange. Roberto would get into the car in the early hours, it would start moving, and his reaction was to grab a non existent steering wheel. He would tense up in the front seat, observe every single object moving past, and hope the vehicle knew what it was doing. He always arrived safe, and day by day comfort washed over him.

By the end of week one, the routine was sorted. Make some coffee, put it in the cup holder thermos. Have a second pair of clothes in the passenger’s seat. Keep the pajama, and snuggle in the comfy blanket, and let the car drive the 2h into the factory. Wake up, change, get the coffee, and walk into work feeling energized! How amazing this technology was!

All the happiness of having a car went out the window when one of Roberto’s colleagues had an accident at the factory. Years of cutting costs, old equipment, and negligence cost a friend his life. The unhappiness was generalized. People wanted to get together and complain — protest if you will.

Unfortunately, as soon as corporate heard about this, some people’s cars started not driving to the plant. They were never officially reprimanded for missing work, they just would not show up. They were told to enjoy the day off, on company pay. Which they did, but at what cost?

I guess investing in all those cars did pay off. No unionizing, no protests. Or at least, when they did happen, some of the key charismatic players were easily moved aside.

And the world continued moving forward.

Leave late, arrive early. The DoubleFast way.

cars that can go on the fast lane — subscription model

Jan had been a longtime customer of DoubleFast for years now. She was racking in all the rewards. Discounted memberships, car upgrades, extra comfort during service stops, the list went on. Climbing the corporate ladder came not only with a good paycheck, it brought these perks with it. Well deserved for all the hard work and time she put in for the well being of the company. Time was valuable, and with DoubleFast she could leave her house at 8am and be in the office by 9am — for the first meeting of the day.

The new corporate guy was there. Jim. He looked tired. Unlike Jan, he couldn’t yet afford the luxury of DoubleFast. He actually lives close to Jan, but his self-driving car isn’t upgraded, therefore he needs to use the regular lanes. The traffic is insane, it actually takes 2h30 to get to the office. And sleeping in the car just isn’t the same as sleeping in your own bed and making it to work in slightly under 1h.

So what is DoubleFast?

DoubleFast is what years of lobbying around infrastructure do to you. Verizon, Comcast, and others tried to do this with the internet. They tried to remove Net Neutrality, and give better bandwidth and speed to some big players. Well, unfortunately for them, the internet is jam packed with smart people that don’t need to leave their homes to fight — so they turned to infrastructure. No one cares about infrastructure.

Grease the palms of the logistics industry leaders, some other big names in the transporting world, and sure enough you get to build real fast lanes in the real world. Just enough to make a few extra millions, not enough to cause a big riot. Just enough perks to get people to play the game, and defend what you are doing. It’s sick. But it is the world Jan and Jim live in today — and it sucks.

The end. These are just some half baked thoughts about things that could go wrong. Maybe they are unrealistic, maybe they aren’t. What matters is we should keep our eyes open. New technologies will certainly bring creative ways to make inequality larger. Lets not be blinded by shiny lights and perks.

Part of a good product is to be able to provide the users with a good experience while they use it, or develop tools that interact with it. To do so, you need to be able to tell them exactly what is going wrong. A simple 400 Bad Request won’t work, you need to be able produce custom error messages and codes.

However, what should be an easy task, isn’t. Flask-Restplus (or at least the fork we use internally) doesn’t seem to support raising custom Exceptions for JSON payloads that are sent to a POST endpoint. So how did we solve this problem?

Internally, when a request is made to an endpoint, the dispatch_request will handle it. Let’s look at it:

Dispatch Gist

We can see that after some logic, the method self.validate_payload is called. By inspecting the code a bit more we can understand that the method that needs to be overwritten is the validate method that exists inside the ModelBase class. Investigating this method shows that Flask-Restplus is internally using jsonschema to validate the payloads being sent to an endpoint.

JSONschema is a very useful way of defining properties of a certain json object.

We want to leverage this to:

1. Prevent our payload of taking any extra fields
2. Prevent certain properties from taking unexpected types a. i.e. an id key is a string and not a boolean

So lets start!

First we define a new model that inherits from the Flask-Restplus model. To solve point 1. in our list, we want to add the JSONschema property “additionalProperties”: false to the model’s schema.

Validation Model Gist

This makes sure that the property we added is always passed to copies of the object. This is something that happens internally with Flask-Restplus that I haven’t had time to investigate properly.

The next step is to validate the entire JSON payload. To do so we override the validate method as we concluded in the beginning.

Validate Gist

Inside the _convert_type_error_to_custom function, you can then do the logic that handles the jsonschema error that is returned, and return the appropriate Exception. Personally, I passed a list of exceptions to the MultipleErrorException to return all errors to the user with only one call of the endpoint.

For practicality when registering the models, and remaining Flask-esque you can create your own Namespace class too.

Validation Namespace Gist

A problem that you will face is that the validation will be done before the authorization for the user is checked. This is a problem because an unauthorized user can sniff out the payload structure of your endpoint.

To avoid this, you can simply do the following:

Authorized Resource Gist

Hope this helps!

Thanks for reading, Jose :)

Originally published at joseantunes.tech.

I am writing this post to reflect on my career so far, and how I got to where I am.

The situations I have been through have shaped me, my decision process, and how I behave in a team/company. I hope that this text helps whoever reads it — how a guy that studied physics got into the startup world, and made something out of it.

I started my career in tech at CognitionX. I worked for two weeks at Zomato in Portgual, as a data collector, and that attracted CognitionX to my profile. Barely did they know that I was doing manual data collection… I was hired to be a Data Miner, and I was lucky enough that I knew how to code a little bit in Python.

I used the little code that I knew to get results, and tried to learn as much as possible on the job.

First lesson: Leverage the people around you to become better at what you do. You are the average of the five people you spend the most time with.

I was so lucky to be surrounded by great people.

Great not only in the sense that they knew what they were doing, great also in the sense that they were genuine people and wanted to help others. So what were the things that I learned at my first job?

• Working properly with Git;
• Understand Python to a deeper level;
• Understand more about how to write software;
• Learn about unit testing (I know what you are thinking…);
• Learn about how businesses operate, and their issues;

All these things were just the beginning of my journey. Every time I got a step further, I could see more things that I needed to learn in the distance.

As I went on, I learned better scraping techniques from an incredible data miner that we hired, but he was so good I was able to focus on being a Backend Engineer. Hence I started building a REST API with the backend team.

Why did I switch to backend? I have a fear of being outdated, and I want to be able to build enourmous, resilient data pipelines. For the future, for AI applications… I had to start somewhere.

During this time I worked closely with a very experienced dev and a friend that was on a journey to learn — just like me. This casual team of 3 used Flask first, then Django to build an API — and powered some of its functionalities with PostgreSQL, ElasticSearch and Neo4j. It was a time of immense productivity and learning.

Eventually problems around the business, and the way people were dealt with drove me away, took away my passion for CognitionX, and I moved on. I decided to move to Cytora, as I perceived it would be a great place to learn and give the next steps in my young career.

From the outside Cytora seemed to have a working product, that was delivering value to customers. That posed challenges that I hadn’t faced before.

• How do you maintain an app that is in production?
• How do you add features without breaking the app for existing customers?
• What does good product management look like?
• Am I good enough to deliver value to new team fast, when different systems are in place?

Over the first six months, I definitely learned a lot regarding all these points. I was able to adapt, to communicate accordingly, to contribute. I coded fast, and I got stuff done. I thought I was doing really well, but looking back, I definitely made mistakes.

First Mistake: I always deferred decisions to my team mates due to their computer science background, and downplayed my experience and the things I had learnt. This meant that over time my opinion was getting less valuable. I should have engaged in dialogue more, and have debated my points of view more often, as well as disagreeing with people until we reached a shared understanding.

Second Mistake: A good developer isn’t the one that codes the fastest. A good developer writes good tickets to represent work, thinks appropriately about the solution, implements it, and documents it well. He also writes good tests, communicates effectively with colleagues, and helps them. Being busy isn’t good, you can become a bottleneck, or stop performing to the best of your ability.

My time at Cytora is divided in two parts though.

In this initial part we were managing a monolithic API, and in the second part our new director of engineering showed us how to work with microservices, in a distributed architecture.

What I learned over this year was mind blowing:

• Trunk-based development;
• How do you do proper continuous deployment;
• What are component tests, smoke tests, stress tests, load tests, acceptance tests;
• How to trust your system, do proper monitor and logging, and let go of the idea you can understand it all;
• Golang — yes, I had to learn it;
• Kaban training was really useful;
• How to design new systems and decouple them;

In the end, I am not even close to knowing all that I want and need to know. I still have much to learn.

I have left Cytora (that is a story for another time), and am taking my chances with Splend. I will be learning a lot on the architecture side of things, and building a complete system. I will have input in the culture, and how we digitally transform the company.

I hope I am making the right choice for my future.

Thanks for reading,

Jose

Originally published at https://joseantunes.tech.

If you are a developer in today’s day and age, you might have requested to get a Dell XPS laptop. Of course you asked for the Ubuntu developer version. Hopefully you got it.

If you didn’t get it, and got the Windows version instead, you installed Ubuntu, and probably no longer have Wifi. That is what happened to me. Lets solve it.

To start, when the laptop is booting, go into the BIOS and make sure you get the model right. After that, with the model number, go into Dell’s website and get the specs — precisely the Wireless card model.

In my case, it said the Wireless card was a Killer AX1650W but it said on every Ubuntu menu Intel Corporation. This is because the Killer card was a partnership with Intel I assume. So don't lose your mind thinking that you got a different card. Just follow all the stackoverflow tutorials.

The best is to check the Killer website though: here.

Follow the steps, reboot, and you are done. Wifi is back!

Thanks for reading,

Jose

Originally published at https://joseantunes.tech.