<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Surya Sankar on Medium]]></title>
        <description><![CDATA[Stories by Surya Sankar on Medium]]></description>
        <link>https://medium.com/@suryasankar?source=rss-aad6b702d70a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HIDGhuD_Zi0IUhfC920mCQ.jpeg</url>
            <title>Stories by Surya Sankar on Medium</title>
            <link>https://medium.com/@suryasankar?source=rss-aad6b702d70a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 04:13:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@suryasankar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Prisma + NextAuth setup for NextJS]]></title>
            <link>https://suryasankar.medium.com/prisma-nextauth-setup-for-nextjs-209dfe1b3293?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/209dfe1b3293</guid>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[nextauth]]></category>
            <category><![CDATA[prisma]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Thu, 13 Nov 2025 00:00:13 GMT</pubDate>
            <atom:updated>2025-12-21T14:17:52.342Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/851/0*TuIkXhKU0deebWaY" /></figure><p>I have been building <a href="https://toolgebra.com/">Toolgebra</a> primarily as a client side utilities website. But recently I have also started building features where the utilities can be chained together into workflows. Now these workflows need to be saved of course, and that is where the need for authentication and database persistence comes in. In my previous projects I have used django + postgres + Auth0 for this. But since I am more serious about Toolgebra, I want a cost friendly solution to host and acquire users. A separate django service would incur separate hosting costs. Reusing the same NextJS app for the backend as well seemed like a better choice. So I did some research and decided to use</p><ol><li><a href="https://authjs.dev/reference/nextjs">NextAuth</a> as the auth framework</li><li><a href="https://www.prisma.io/docs">Prisma</a> as the ORM and</li><li><a href="https://neon.com/">Neon</a> as the hosted Postgres solution</li></ol><p>Though each of these seem to be fairly popular choices in the NextJS ecosystem, combining them all together took me nearly a weekend full of work as the documentation was all over the place. I am recording it as a blog post here for posterity. This setup works as of Nov 2025, but if you are reading this later, it is best to refer the latest docs of these services once again.</p><h3>Installing Prisma</h3><p>Installing Prisma itself is quite straightforward. You can just follow the <a href="https://www.prisma.io/docs/guides/nextjs">guide for Nextjs</a>. By default it shows the instructions for configuring Prisma Postgres — which is their hosted postgres solution. If you are not looking to use it, you can skip the installation of @prisma/extension-accelerate</p><p>To install just the packages necessary for using prisma with any postgres installation, just run these</p><pre>npm install prisma tsx --save-dev npm install @prisma/client dotenv</pre><p>After installing it, you can run</p><pre>npx prisma init</pre><p>The installation mentions a --db argument which you can skip if you are not using their hosted postgres solution.</p><p>Once you have initiated, you can start the local database server automatically created by prisma by running</p><pre>npx prisma dev</pre><p>The connection to this DB server should have already configured been by the preceding init script which would have setup the DATABASE_URL var in your .env file. (If you are running your own local postgres db already, you can of course replace this DATABASE_URL with a connection string to that local db)</p><p>After this, the rest of their <a href="https://toolgebra.com/blog/(https://www.prisma.io/docs/guides/nextjs)">guide</a> is about how to setup a functioning NextJS app using Prisma. If you already have an app into which you are integrating Prisma, you can skip it and jump to the NextAuth part. For the purposes of this tutorial, let’s continue to follow the guide and setup the example Posts app they have described.</p><h3>Setting up the example app</h3><p>First, add the models to the schema.prisma file as mentioned <a href="https://www.prisma.io/docs/guides/nextjs#22-define-your-prisma-schema">here</a> and then add dotenv as mentioned <a href="https://www.prisma.io/docs/guides/nextjs#23-add-dotenv-to-prismaconfigts">here</a></p><p>Then seed the database like mentioned <a href="https://www.prisma.io/docs/guides/nextjs#25-seed-the-database">here</a>, but you will need to change the first import line in the seed script to use import from prisma/client instead of just prisma</p><pre>import { PrismaClient, Prisma } from &quot;../app/generated/prisma/client&quot;;</pre><p>Note that you can also use @/app instead of ../app.</p><p>Then setup lib/prisma.ts as mentioned <a href="https://www.prisma.io/docs/guides/nextjs#26-set-up-prisma-client">here</a>. But here as well you will need to change the first import line to</p><pre>import { PrismaClient } from &#39;../app/generated/prisma/client&#39;</pre><p>For some reason the prisma guide there mixes between src/app convention and app/ convention alternatingly. If you have chosen to use src/ folder as the root in nextjs repo, then use the ../src/app/generated path. If not it will be ../app/generated.</p><p>Then add the different pages mentioned in the guide — <a href="https://www.prisma.io/docs/guides/nextjs#3-query-your-database-with-prisma-orm">home</a>, <a href="https://www.prisma.io/docs/guides/nextjs#4-add-a-new-posts-list-page">posts listing</a>, <a href="https://www.prisma.io/docs/guides/nextjs#5-add-a-new-posts-detail-page">post detail</a>, <a href="https://www.prisma.io/docs/guides/nextjs#6-add-a-new-posts-create-page">new post form</a></p><h3>Setting up NextAuth</h3><p>If prisma setup is done, we can move on to configuring NextAuth. For NextAuth there are 2 sets of docs available. At <a href="https://next-auth.js.org/getting-started/introduction">NextAuth website</a> and in <a href="https://authjs.dev/reference/nextjs">Auth JS website</a>. The AuthJS one is the newer one and we can follow that.</p><p>Let’s follow the steps there. You can first install the package</p><pre>npm install next-auth@beta</pre><p>Then run npx auth secret. This adds AUTH_SECRET var in .env.local. Since we are already using .env, we can just continue using that. Copy over this variable from .env.local to .env and delete the .env.local file.</p><p>Then you can just follow the steps <a href="https://authjs.dev/getting-started/installation?framework=Next.js">here</a>.</p><p>Add auth.ts</p><pre>import NextAuth from &quot;next-auth&quot;<br> <br>export const { handlers, signIn, signOut, auth } = NextAuth({<br>  providers: [],<br>})</pre><p>And then ./app/api/auth/[...nextauth]/route.ts</p><pre>import { handlers } from &quot;@/auth&quot; // Referring to the auth.ts we just created export const { GET, POST } = handlers</pre><p>But skip ./middleware.ts since it is no longer <a href="https://nextjs.org/docs/messages/middleware-to-proxy">supported by nextjs</a></p><h3>Configure Google oauth</h3><p>Go to <a href="https://console.cloud.google.com/auth/clients">Google Cloud Console</a> and setup a Web Client. Set the origin to localhost:3000 and callback url to localhost:3000/api/auth/callback/google</p><p>Then follow the steps for Google auth <a href="https://authjs.dev/getting-started/authentication/oauth">here</a>. Add components/sign-in.tsx as mentioned there.</p><pre>import { signIn } from &quot;@/auth&quot;<br> <br>export function SignIn() {<br>  return (<br>    &lt;form<br>      action={async () =&gt; {<br>        &quot;use server&quot;<br>        await signIn()<br>      }}<br>    &gt;<br>      &lt;button type=&quot;submit&quot;&gt;Sign in&lt;/button&gt;<br>    &lt;/form&gt;<br>  )<br>}</pre><p>We can also add components/sign-out.tsx as mentioned <a href="https://authjs.dev/getting-started/session-management/login#signout">here</a></p><pre>import { signOut } from &quot;@/auth&quot;<br> <br>export function SignOut() {<br>  return (<br>    &lt;form<br>      action={async () =&gt; {<br>        &quot;use server&quot;<br>        await signOut()<br>      }}<br>    &gt;<br>      &lt;button type=&quot;submit&quot;&gt;Sign Out&lt;/button&gt;<br>    &lt;/form&gt;<br>  )<br>}</pre><p>The guide doesn’t mention anything about where to add these signin and sign out components. So let’s add a navigation component</p><pre>import Link from &quot;next/link&quot;;<br>import { auth } from &quot;../auth&quot;<br>import SignIn from &quot;./sign-in&quot;;<br>import SignOut from &quot;./sign-out&quot;;<br><br>export default async function Navigation() {  <br>    const session = await auth();<br><br>    return (<br>        &lt;&gt;<br>            &lt;nav className=&quot;sticky top-0 z-50 bg-white/80 backdrop-blur-md border-b border-gray-200&quot;&gt;<br>                &lt;div className=&quot;max-w-7xl mx-auto px-4 sm:px-6 lg:px-8&quot;&gt;<br>                    &lt;div className=&quot;flex items-center h-16&quot;&gt;<br>                        {/* Logo */}<br>                        &lt;Link href=&quot;/&quot; className=&quot;flex items-center space-x-2&quot;&gt;<br>                        &lt;span className=&quot;text-xl font-semibold text-[#369b6f]&quot;&gt;<br>                            Prisma + NextAuth + Neon<br>                        &lt;/span&gt;<br>                        &lt;/Link&gt;<br><br>                        {/* Spacer */}<br>                        &lt;div className=&quot;flex-1&quot;&gt;&lt;/div&gt;<br><br>                        {/* Navigation Links */}<br>                        &lt;div className=&quot;hidden md:flex items-center mr-8&quot;&gt;<br>                        &lt;Link<br>                            href=&quot;/posts&quot;<br>                            className=&quot;text-gray-700 font-medium hover:text-[#369b6f] transition-all duration-200&quot;<br>                        &gt;<br>                            Posts<br>                        &lt;/Link&gt;<br>                        &lt;/div&gt;<br><br>                        {/* Auth Buttons */}<br>                        &lt;div className=&quot;flex items-center space-x-2 sm:space-x-4&quot;&gt;<br>                        {session?.user ? (<br>                            &lt;SignOut/&gt;<br>                        ) : (<br>                            &lt;SignIn/&gt;<br>                        )}<br>                        &lt;/div&gt;<br>                    &lt;/div&gt;<br>                &lt;/div&gt;<br>            &lt;/nav&gt;<br>        &lt;/&gt;<br>    );<br>}</pre><p>And then include it in app/layout.tsx</p><pre>export default function RootLayout({<br>  children,<br>}: Readonly&lt;{<br>  children: React.ReactNode;<br>}&gt;) {<br>  return (<br>    &lt;html lang=&quot;en&quot;&gt;<br>      &lt;body&gt;<br>          &lt;Navigation/&gt;<br>          &lt;main&gt;<br>            {children}<br>          &lt;/main&gt;<br>      &lt;/body&gt;<br>    &lt;/html&gt;<br>  );<br>}</pre><p>The app should now show a navigation bar at the top with the Sign in / Sign out buttons. If you are adding NextAuth to your own app, you can similarly add it somewhere in your navigation like this. But note that the sign in and sign out components we have coded now are server side components. They cannot be used in your client components. We will explore how to do that in a future post.</p><h3>Integrate Prisma with NextAuth</h3><p>Now that we have the auth working, we need to start persisting the authentication. For that we are going to use prisma adapter. Let&#39;s first install it</p><pre>npm install @auth/prisma-adapter</pre><p>Then modify auth.ts to include prisma adapter</p><pre>import NextAuth from &quot;next-auth&quot;<br>import Google from &quot;next-auth/providers/google&quot;<br>import { PrismaAdapter } from &quot;@auth/prisma-adapter&quot;;<br>import prisma from &quot;@/lib/prisma&quot;;<br><br>export const { handlers, signIn, signOut, auth } = NextAuth({<br>    adapter: PrismaAdapter(prisma),<br>    providers: [Google],<br>})</pre><p>Now we need to add the schema needed by NextAuth. We can copy it from <a href="https://authjs.dev/getting-started/adapters/prisma#schema">here</a>. Note that we had already created an User table while creating the example app earlier. Now that will need to change as NextAuth uses a string column for user id and adds a few more columns to the user table. Since the schema will change for the User table, it is simpler to drop the current database and recreate it. (But if you already have a functioning app with an User table, you will need to research more on how to integrate NextAuth safely into your schema. That is outside the scope of this tutorial. )</p><p>To do this, let’s first remove the seed script so that it doesn’t see data again. Remove this line from prisma.config.ts</p><pre>seed: `tsx prisma/seed.ts`</pre><p>Then run this to drop the current tables.</p><pre>npx prisma migrate reset</pre><p>and then run this to create the new table structure</p><pre>npx prisma migrate dev</pre><p>After this, if you restart the dev server with npm run dev and do a signin, you should see a new entry created in the User table. You can verify that by viewing the table contents using <a href="https://www.prisma.io/docs/orm/tools/prisma-studio">prisma studio</a> by running npx prisma studio</p><p>Now that we have a functioning setup for NextAuth + Prisma, let’s complete one final step in our example app just to complete this tutorial. Let’s configure the Post creation page to send session.userId as authorId instead of the hard-coded authorId we had copied from the Prisma tutorial.</p><pre>export default async function NewPost() {<br>  const session = await auth();<br><br>  async function createPost(formData: FormData) {<br>    &quot;use server&quot;;<br><br>    if(!session?.user?.id){<br>      return;<br>    }<br><br>    const title = formData.get(&quot;title&quot;) as string;<br>    const content = formData.get(&quot;content&quot;) as string;<br><br>    await prisma.post.create({<br>      data: {<br>        title,<br>        content,<br>        authorId: session.user.id,<br>      },<br>    });<br><br>    revalidatePath(&quot;/posts&quot;);<br>    redirect(&quot;/posts&quot;);<br>  }<br><br>  if(!session?.user){<br>    return &lt;div&gt;You need to sign in to create a post&lt;/div&gt;<br>  }<br><br>  ...<br><br>}</pre><h3>Conclusion</h3><p>That concludes the tutorial. We mostly followed the already available official guides. But they don’t smoothly work together out of the box and we have to make various adjustments — which I figured out with a lot of trial and error. Hope this post saves you a couple of days of effort.</p><p>At the start of the post, I had mentioned that I used Neon as the hosted postgres solution to deploy <a href="https://toolgebra.com/blog/toolgebra.com">Toolgebra</a> on the cloud. I have published a separate post on <a href="https://toolgebra.com/blog/prisma-neon-setup-for-nextjs">how to integrate Prisma &amp; Neon</a>. Please check that out as well.</p><p><em>Originally published at </em><a href="https://toolgebra.com/blog/prisma-nextauth-setup-for-nextjs"><em>https://toolgebra.com</em></a><em> on November 13, 2025.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=209dfe1b3293" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Publish a JS and CSS Library to NPM Using Rollup]]></title>
            <link>https://medium.com/geekculture/how-to-publish-a-js-and-css-library-to-npm-using-rollup-5406dbee51fa?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5406dbee51fa</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[npm]]></category>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[rollup]]></category>
            <category><![CDATA[webpack]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Sat, 03 Apr 2021 08:04:54 GMT</pubDate>
            <atom:updated>2021-04-07T21:50:26.676Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*f-esfxHEURcTs5iW.png" /></figure><p>In this guide, we will discuss how we can create a javascript library and publish it to npm. In a <a href="https://suryasankar.medium.com/a-simple-nested-tree-menu-with-vanilla-js-and-css-d30159f14096">previous post</a> we had built a small script for creating a nested tree menu. We will convert that into a npm package here.</p><h3>Step 1: Initialize the directory and the repo.</h3><p>Let’s create a folder for the library and add a README.md file there. This would be a good time to do git init and create a remote github repo to track it as well. (If you are new to this, you can check github howto <a href="https://docs.github.com/en/github/importing-your-projects-to-github/adding-an-existing-project-to-github-using-the-command-line">here</a>)</p><h3>Step 2: Add scripts and stylesheets inside the src folder</h3><p>In this guide we intend to build the package and then use it from an example project. We will create 2 separate folders. One for the package and one for the example.</p><p>Inside the package directory we will create separate folders for the source files and the built files. In modern javascript workflow, you rarely distribute the handcoded source files directly. The source files are passed through bundlers which perform various transformations and optimizations and provide output files which can then be used in the browser or other environments. So we will go ahead and create two folders names src - for the source files and dist - for the output files. The dist folder need not be touched by us. It will be populated by the bundler.</p><p>Inside the src folder let’s create 2 subfolders — js and styles to hold the javascript files and the stylesheets respectively.</p><p>The folder structure would look like this now</p><pre>├── README.md<br>├── examples<br>│   ├── dist<br>│   ├── src<br>└── package<br>    ├── dist<br>    └── src<br>        ├── js<br>        └── styles</pre><p>We will copy the contents of the style tag in our <a href="https://techonometrics.com/posts/a-simple-nested-tree-menu-with-vanilla-js-and-css">nested tree menu example</a> to package/src/styles/main.scss. We are using the scss extension so that we can use sass specific features in the future. But for now, our stylesheet is actually just pure css only.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7ea3fd2cd34b47b25b9978e37d298dd5/href">https://medium.com/media/7ea3fd2cd34b47b25b9978e37d298dd5/href</a></iframe><p>Similarly we will also copy the content of the style tag into package/src/js/index.js.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c78ee16531dd518452141b0d85a0df4/href">https://medium.com/media/4c78ee16531dd518452141b0d85a0df4/href</a></iframe><p>We have made two changes to the content copied from the script tag.</p><ol><li>We are importing the main.scss at the top. This will tell our bundler how to locate the stylesheets so that they can be minified.</li><li>We have added an export default listree statement at the bottom, which lets us export our function as an ES module.</li></ol><h3>Step 3: Initialize the npm package</h3><p>Now let’s call npm init from inside the package folder. This will create a package.json which we will edit to modify the main key and add 2 more keys - module and browser. We will also set the files property to whitelist the dist folder to be added to the package.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c19891783ee60fc155d92c55e54b023b/href">https://medium.com/media/c19891783ee60fc155d92c55e54b023b/href</a></iframe><p>The main key represents the entrypoint of the package. If we were building a library which was meant to be used on the server side using NodeJS, we would point this to a CommonJS output format. But here we are building a purely frontend library which has no use outside the browser context. So we will set it to point to the <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules">ES module</a> format output - which would make it accessible via import statements.So the value is set to dist/listree.esm.min.js. The file doesn&#39;t exist yet and we will generate it using our bundler in the subsequent steps.</p><p>The module entry point is meant to be used to point to ES module output specifically. This is useful in the cases when the main is set to point to a CommonJS output. In our case we pointed main already to an .esm file. But anyway we will set the module property also to point to dist/listree.esm.min.js.</p><p>The browser entry should point to the script file that can be loaded directly in a browser script tag. We will set this to dist/listree.umd.min.js where umd stands for universal module definition.</p><p>Both the entrypoint files — esm as well as umd will be generated using a bundler as follows</p><h3>Step 4: Install rollup</h3><p>We are going to use rollup as the bundler for generating our package distribution files. There are various bundling systems available — webpack being the most popular right now. But rollup is more suited for generating redistributable libraries.</p><pre>npm install --save-dev rollup rollup-plugin-scss rollup-plugin-terser</pre><p>Apart from rollup, we are also installing 2 more plugins here — one for converting scss to css and the terser plugin for minimizing the generated files</p><h3>Step 5: Create a rollup config js file</h3><p>We instruct the rollup bundler using a rollup.config.js file</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bc6baa2646b0ab32ebcc686658564690/href">https://medium.com/media/bc6baa2646b0ab32ebcc686658564690/href</a></iframe><p>In the above config, we are specifying the input as the index.js we had created inside src/js. We don’t have to specify a separate input for css since we have already imported the styles inside index.js itself.</p><p>We have also configured 2 different outputs, each of which corresponds to one of the entrypoints specified in package.json. Just by mentioning the formats like above, rollup will take care of converting our source into that format.</p><p>We also need to minify the generated output files for distribution, which is done by the terser plugin. And finally we generate a minified output css by using the scss plugin.</p><h3>Step 6: Build the output files</h3><p>Now that all the configuration is done, we can call the rollup command line script with various options. Calling rollup -c will build the package. But instead of directly calling it, we will instead set the commands corresponding to various scripts in the package.json itself by specifying a scripts array as follows</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d0b13012169807ca8e28b95201673fc4/href">https://medium.com/media/d0b13012169807ca8e28b95201673fc4/href</a></iframe><p>Now we can just call npm run build and it will generate all the output files inside dist/ folder. We now need to verify it works</p><h3><strong>Step 7: Testing the umd version of the package with an example html by using npm link</strong></h3><p>In order to check how the package works when it is imported by another application, we can create a sample application and import this package.</p><p>We will create a file umd.html inside the examples folder. Let&#39;s copy the following html content in the file</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dda19164c911397b681ba74d35dd202f/href">https://medium.com/media/dda19164c911397b681ba74d35dd202f/href</a></iframe><p>Note that the file above references the js and css files from node_modules. But we haven&#39;t published our package to npm yet. So how do we make sure that the node_modules folder inside examples will contain the listree package?</p><p>NPM provides the command npm link just for this. If you are familiar with python&#39;s package environment, this has the same effect as running python setup.py develop from inside a package folder. It essentially installs the package in whichever place you want to test it from - in the development mode. So if any change is made in the package code, that will be instantly available for the invoking application to test as well. This ensures that we don&#39;t have to publish every time we want to test a small change we have made. So what we are going to do instead is to link the package locally.</p><p>So first we will run npm link inside our package folder. Once this command succeeds the package is linked to the global node modules folder in the development machine.</p><p>Now in order to use our package in another package that we are developing, we will go inside the examples folder and run npm link listree there. This will create a node_modules folder inside the examples folder with listree package included inside. On opening the examples/umd.html file in browser, we can see that the functionality works as expected.</p><h3>Step 8: Testing the import of the package as an ES6 module by using webpack</h3><p>In order the check the package’s behavior as an ES6 module, we will create a src/index.js file inside the examples folder with the following content</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/94b8042b095b283e729859f7125dd8a5/href">https://medium.com/media/94b8042b095b283e729859f7125dd8a5/href</a></iframe><p>In order to make this script loadable via a script tag, we will use webpack to transform and bundle it. We use webpack instead of rollup here because webpack is more suited for application bundling out of the box (while rollup is more suitable for library bundling) and it is more appropriate to test our package’s behavior when it is bundled via webpack since webpack is the most commonly used package bundler by web developers.</p><p>We will first have to install webpack and webpack-cli by running npm install -g webpack webpack-cli</p><p>Then we add a webpack.config.js like this</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1c6802872cdbe9f826ff5d2582b6ed1f/href">https://medium.com/media/1c6802872cdbe9f826ff5d2582b6ed1f/href</a></iframe><p>If we run webpack, it will create a dist/mylistree.js file. We will now copy our umd.html to examples/es.html and replace the script tag at the bottom which refers to the listree.umd.min.js with a script tag which refers this dist/mylistree.js file. That is, we will replace this</p><pre>&lt;script src=&quot;./node_modules/listree/dist/listree.umd.min.js&quot;&gt;&lt;/script&gt;<br>&lt;script&gt;<br>   listree();<br>&lt;/script&gt;</pre><p>with this -</p><pre>&lt;script src=&quot;dist/mylistree.js&quot;&gt;&lt;/script&gt;</pre><p>In the second case, we don’t need to invoke the listree() function as we have already invoked it inside our index.js which got bundled as mylistree.js. Now if we open examples/es.html in the browser, we can see that the functionality works as expected.</p><p>Note that we have checked both the html files by loading them directly using file:// protocol. If we want to test them in the context of them being served via a webserver, we can use webpack&#39;s html and devserver plugins to do the same. But let&#39;s keep that as a task for a future post for the sake of brevity.</p><h3>Step 9: Push to github repo</h3><p>Now that we have completed creating the package, let’s also do a push to github. We also need to edit the .gitignore to exclude the dist and node_modules folders.</p><pre>node_modules/<br>dist/</pre><p>It is ok to version commit the package-lock.json file though.</p><h3>Step 10: Publish the package to npm</h3><p>To do this, we will first create an account on <a href="https://www.npmjs.com/">https://www.npmjs.com/</a> ( only if we don’t already have one). Then on the local development machine, we execute the command npm login. This will ask us to enter the username, password and email. Once entered, we are ready to publish the package.</p><p>By just calling npm publish inside the package folder and we can see the package getting published to the npm repository.</p><h3>Step 11: Use the published package in the html</h3><p>A great thing about publishing to npm is that our package gets CDN support out of the box. Publishing a package to npm makes it automatically available via various content delivery networks like <a href="https://unpkg.com/">Unpkg</a>, <a href="https://cdnjs.com/">CDNJS</a> and <a href="https://www.jsdelivr.com/">JSDelivr</a>. We can choose any of these. For example, if we decide to use unpkg, we can access the umd js version of our package like this <a href="https://unpkg.com/listree/dist/listree.umd.min.js">https://unpkg.com/listree/dist/listree.umd.min.js</a>. We can test this by modifying both the css and the js tag urls in our example html files to point to this CDN links instead of the local files, and we will see that the functionality still works as expected.</p><p>Thus we have created and published a package to NPM repository and successfully used the same in an application. That concludes this tutorial. The code of the package created for this tutorial is available at <a href="https://github.com/SuryaSankar/listree">https://github.com/SuryaSankar/listree</a>. Please consider starring the repo if you find the package or this tutorial useful. The package itself is available for use in npm under the name listree <a href="https://www.npmjs.com/package/listree">here</a>. You can install and use it by just calling npm install listree in your project.</p><p><em>Originally published at </em><a href="https://techonometrics.com/posts/how-to-publish-a-js-and-css-library-to-npm-using-rollup/"><em>https://techonometrics.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5406dbee51fa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/geekculture/how-to-publish-a-js-and-css-library-to-npm-using-rollup-5406dbee51fa">How to Publish a JS and CSS Library to NPM Using Rollup</a> was originally published in <a href="https://medium.com/geekculture">Geek Culture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A simple nested tree menu with vanilla js and css]]></title>
            <link>https://suryasankar.medium.com/a-simple-nested-tree-menu-with-vanilla-js-and-css-d30159f14096?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/d30159f14096</guid>
            <category><![CDATA[html]]></category>
            <category><![CDATA[css]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[ui]]></category>
            <category><![CDATA[web-design]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Sun, 07 Mar 2021 11:45:41 GMT</pubDate>
            <atom:updated>2021-03-07T11:45:41.830Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*lmYm5aKr4mRYK_Rv.gif" /></figure><p>While building a dashboard layout, I recently had a requirement for a nested tree menu. It would preferably have to support any arbitrary level of nesting. There were some packages available already like <a href="https://www.jstree.com/">JSTree</a> — but they were built to support a lot of extra functionalities like changing the menu theme, adding branches dynamically etc which made them bigger than I needed them to be. I didn’t need any of those features and hence decided to build it myself. This short post is intended to document the same</p><h3>Designing the html structure</h3><p>The default choice for any menu type structure is to go with unordered lists. We can do the same here. We just need to make sure that the structure supports deep nesting and sub-headings for sub-menus. At the bare minimum it would look like this</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d13183c3a0eae3fac2ab5d516ebeb587/href">https://medium.com/media/d13183c3a0eae3fac2ab5d516ebeb587/href</a></iframe><h3>Making it interactive using css and js</h3><p>Here we implement the styling and scripts that would make this a proper nested menu</p><h4>Step 1: Remove the default ul styling</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f2f5c9213b6561de2f166b77cc274df7/href">https://medium.com/media/f2f5c9213b6561de2f166b77cc274df7/href</a></iframe><h4>Step 2: Add a click handler which expands and collapses the submenu</h4><p>We do this by targeting all the elements with the submenu heading’s class name and then setting it’s sibling element’s (which would be the submenu ul) display to none (to hide it by default). Then we add a click handler on the submenu heading div which toggles the sibling submenu ul’s display property between none and block</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7966d8f344bfa4be5628e3210a04ef4c/href">https://medium.com/media/7966d8f344bfa4be5628e3210a04ef4c/href</a></iframe><h3>Step 3: Add + and — markers next to the headings to denote its state</h3><p>The menu headings should show a plus next to them when they are in a collapsed state indicating that they can be expanded and a minus next to them when they are expanded.</p><p>To do this we first add two classes — collapsed and expanded on the submenu heading which add the characters + and -</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5860190193e6a4d5cb56b0be30fb1751/href">https://medium.com/media/5860190193e6a4d5cb56b0be30fb1751/href</a></iframe><p>And then we add the logic to add and remove the classes to the event handler</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c5c700f1eb34d61daf862c8f8330e946/href">https://medium.com/media/c5c700f1eb34d61daf862c8f8330e946/href</a></iframe><h3>Step 4: Add a submenu boundary when it is expanded</h3><p>A final touch would be to add a dotted boundary on the left when a submenu is expanded — which lets the user see clearly which items are in the same level of nesting. To do this we add the following css</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d946713148ff8658804d059974b8ff06/href">https://medium.com/media/d946713148ff8658804d059974b8ff06/href</a></iframe><p>Since we have already added the styling and logic for hiding and showing this class, this boundary also will be hidden and shown along with it.</p><p>And that finishes our work. Combining it all together, here is a sample html with the styling and the script which achieves the nested menu functionality. Copy and paste this html and open it via a browser, it will have the functionality demonstrated by the cover image at the top of this post.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1847ba4d807dabe0a8bd7d9b815e2de0/href">https://medium.com/media/1847ba4d807dabe0a8bd7d9b815e2de0/href</a></iframe><p>Now that it is working as expected, we will next see how we can convert this into a reusable library in the next post.</p><p><em>Originally published at </em><a href="https://techonometrics.com/posts/a-simple-nested-tree-menu-with-vanilla-js-and-css/"><em>https://techonometrics.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d30159f14096" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Configuring a Flask app to log user and session ids in nginx logs]]></title>
            <link>https://suryasankar.medium.com/configuring-a-flask-app-to-log-user-and-session-ids-in-nginx-logs-5147253ec397?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5147253ec397</guid>
            <category><![CDATA[nginx]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[logging]]></category>
            <category><![CDATA[flask]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Fri, 05 Feb 2021 10:06:40 GMT</pubDate>
            <atom:updated>2021-02-06T09:21:36.798Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to configure a Flask app to log user id and session id in nginx logs</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XLd8N-ohyaO8K1HTP_dWnQ.png" /></figure><p>Nginx access logs are a very important data source for web traffic analysis. Analysing these logs can provide us information like the frequency of hits to specific urls, traffic from specific ips etc. While this is useful by itself, for it to be a proper analytics datasource, it should also be possible to analyse user behaviour over time — monitoring things like when the user signs up, when the user converts to a paying customer etc using the log entries themselves. In order to do this, we can configure our application server to send user and session id details in every response and then configure nginx to then record those details in the logs.</p><p>Since we want this information to be sent for each and every request, the preferable way to do it would be to include it as a part of the header information for every response generated by the server. Different frameworks have different ways of customising the response headers. Here we will see how to do it in Flask</p><h3>Configuring the application server to set user id and session id in header</h3><p>Flask has a concept called <strong>after_request handler </strong>which is a decorator which will ensure that the decorated function is called after every request. So we will use this feature to modify the response headers.</p><p>We can use an after request handler to set some response headers. For custom headers, the convention is to prefix it with a X. So we can call our headers as <strong>X-Userid</strong> and <strong>X-Sessionid</strong>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/565443eb9eaf5f2ead1cf3ed63e1d718/href">https://medium.com/media/565443eb9eaf5f2ead1cf3ed63e1d718/href</a></iframe><p>In the above example, for X-Userid I am setting the user’s email. Any other unique identifier will do as well. For session id, I am using the unique identifier set by the session handling library.</p><p>Now that our application is setting these headers in the response, how do we configure nginx to read and log them ?</p><p>By default this is what an nginx access log entry looks like.</p><blockquote><em>100.123.123.123 — — [10/Apr/2020:13:07:13 +0000] “GET /api/posts/1?user=2343&amp;utm_source=ads HTTP/1.1” 200 75612 “ </em><a href="https://example.com/posts%22"><em>https://example.com/posts&quot;</em></a><em> “Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36”</em></blockquote><p>The default format shown above is called the combined log format. It is defined as follows</p><blockquote><em>$remote_addr — $remote_user [$time_local] “$request” $status $body_bytes_sent “$http_referer” “$http_user_agent”</em></blockquote><p>We can modify this log format using the <strong>log_format</strong> directive. This directive is documented <a href="https://nginx.org/en/docs/http/ngx_http_log_module.html#log_format">here</a>. As mentioned there, various variables are available at log writing time which can be used to log various types of information.</p><p>In our particular case, we need the variables which contain the info in the two response headers X-Userid and X-Sessionid generated by the application server. Typically the headers in the request are available as variables of the format <strong>$http_&lt;insert_request_header_name&gt;</strong> and the headers in the nginx response are available in variables of the format <strong>$sent_http_&lt;insert_response_header_name&gt;</strong>. However the headers we need are response headers which are not set by nginx, but by the flask application server. How do we access those headers ?</p><p>Since nginx acts as the reverse proxy, the application server (Flask in our case) is considered as an <strong>upstream server</strong> in nginx terminology. And the <a href="https://nginx.org/en/docs/http/ngx_http_upstream_module.html">documentation corresponding to the upstream servers</a> mentions that these headers are made available as embedded variables which are named like <strong>$upstream_http_&lt;insert_header_name_here&gt;</strong> The documentation also mentions that the upper case letters in the header name are converted to lower case and dashes are converted to underscores. So <strong>X-Userid</strong> becomes available as <strong>$upstream_http_x_userid</strong> and <strong>X-Sessionid</strong> becomes available as <strong>$upstream_http_x_sessionid</strong></p><p>Armed with this info, we can now modify the log_format directive to include this info also in the logs. Open <strong>/etc/nginx/nginx.conf</strong> and replace the log_format directive there with the following</p><blockquote><em>log_format extended ‘$remote_addr — $remote_user [$time_local] “$request” ‘ ‘$status $body_bytes_sent “$http_referer” ‘ ‘“$http_user_agent” ‘ ‘userid=”$upstream_http_x_userid” sessionid=”$upstream_http_x_sessionid”’;</em></blockquote><p>Also make sure to modify the access_log directive to use this modified format as follows</p><blockquote><em>access_log /var/log/nginx/access.log extended;</em></blockquote><p>After making these two changes in the http section of nginx.conf, if we restart the nginx server (and the application server as well), we will see that the log entries are recorded with the user information like this</p><blockquote><em>100.123.123.123 — — [10/Apr/2020:13:07:13 +0000] “GET /api/posts/1?user=2343&amp;utm_source=ads HTTP/1.1” 200 75612 “ </em><a href="https://example.com/posts%22"><em>https://example.com/posts&quot;</em></a><em> “Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.132 Safari/537.36” userid=”</em><a href="https://techonometrics.com/cdn-cgi/l/email-protection#ceafacadaa8eb6b7b4e0ada1a3"><em>someemail@abc.com </em></a><em>“ sessid=”4f23b99ce5c298065aeaac569b3a22854a00eb2”</em></blockquote><p>Now with this extra information recorded as a part of the nginx logs, we can incorporate this into our analytics queries in order to obtain various behavioral insights. We will see some examples of such queries in a later post.</p><p><em>Originally published at </em><a href="https://techonometrics.com/posts/configure-flask-to-log-user-session-ids-in-nginx-access-logs/"><em>https://techonometrics.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5147253ec397" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A basic app factory pattern for flask projects]]></title>
            <link>https://suryasankar.medium.com/a-basic-app-factory-pattern-for-production-ready-websites-using-flask-and-sqlalchemy-dbb891cdf69f?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/dbb891cdf69f</guid>
            <category><![CDATA[sqlalchemy]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[flask]]></category>
            <category><![CDATA[boilerplate]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Wed, 15 Apr 2020 01:20:05 GMT</pubDate>
            <atom:updated>2020-04-16T06:32:26.153Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>A very basic app factory pattern for flask projects</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/419/1*K5teGZsxHzreleIAWNRJSg.png" /></figure><p>I am planning to do a series of posts where we will progressively build a demo ecommerce website using Flask, SQLAlchemy and React. The boilerplate repositories prepared for these posts might serve as useful starter kits for others. Plus it will help the reader get a feel of building a production grade website from scratch.</p><p>These posts will be based on some of my learnings obtained from building an ecommerce website for the past 6 years. Since I mostly worked on backend apis, devops and analytics — my posts will be focused more on those areas. But I will also try to post a little on frontend tools as well, though my experience there is comparatively limited.</p><p>In the first post of the series, we are just going to create a bare minimum flask app in the app factory pattern. The objective is just to build a skeleton architecture which would be able to accomodate the various types of services we will experiment with in the future posts.</p><h3>Pre-requisites</h3><p>It is assumed that the reader has a basic working knowledge of Flask and SQLAlchemy. If not, please read through <a href="https://flask.palletsprojects.com/en/1.1.x/quickstart/">Flask Quickstart</a> and <a href="https://flask-sqlalchemy.palletsprojects.com/en/2.x/quickstart/">Flask-SQLAlchemy Quickstart</a> once. These examples use the simple pattern where a flask application is instantiated and routes registered — all in a single module. While this is suitable for very small applications (in terms of lines of code), for most applications we will need a more modular and scaleable architecture when deploying to production. Flask provides various tools to accomplish this. We are primarily interested in Blueprints and the app factory pattern. You should read the official docs explaining <a href="https://flask.palletsprojects.com/en/1.1.x/blueprints/#blueprints">blueprints</a> and <a href="https://flask.palletsprojects.com/en/1.1.x/patterns/appfactories/">app factories</a> as well once.</p><h3>Checking out the boilerplate repo</h3><p>I am planning to write the accompanying examples for all these posts as boilerplates — which can serve as useful starting points for your own apps. The code for the boilerplate app for this particular post is available <a href="https://github.com/SuryaSankar/flask_pattern_minimal_with_auth">here</a>. Please clone it before you start reading this post. Or at least keep the github link open to follow along.</p><h3>Structure of the app</h3><p>The folder structure looks like this</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a38fb8489d624c37ee5527fcdf3a9a9d/href">https://medium.com/media/a38fb8489d624c37ee5527fcdf3a9a9d/href</a></iframe><p>I will cover testing in a separate post. So the above structure does not include tests directory. Also note that the instance folder mentioned above, won&#39;t actually be available when you clone the repo. It is included in .gitignore to keep it out of version control. It is meant to store the non-shareable configuration details of the app (like passwords, secret keys etc)</p><p>Here is the .gitignore used</p><pre>*.pyc<br>__pycache__/<br>instance/</pre><p>Before we start reviewing the code, let’s review the other setup steps required — like installing the packages, setting up the db etc.</p><h3>Setting up mysql</h3><p>For this project we are going to use mysql as the database. You can refer to various tutorials available online for the same.</p><p>For example, you can use this <a href="https://linuxize.com/post/how-to-install-mysql-on-ubuntu-18-04/">tutorial</a> to setup a mysql server on an ubuntu machine, and this <a href="https://www.a2hosting.in/kb/developer-corner/mysql/managing-mysql-databases-and-users-from-the-command-line">tutorial</a> for creating a database and granting access to a particular user. Note the information like db username, password and the database name. You will need that later when creating the config file for the app.</p><h3>Setting up the instance config file</h3><p>Create a folder named instance inside the cloned repo. As mentioned earlier, this will remain outside version control. Then create a file named application.cfg.py like this</p><pre>DB_USERNAME = &#39;admin&#39;<br>DB_PASSWORD = &#39;adminpasswd&#39;<br>DB_SERVER = &#39;localhost&#39;<br>DB_NAME = &#39;flask_boilerplate&#39;</pre><pre>SECRET_KEY = &quot;bf23423e58727fcdfdsgsagg681d9ewrrewr234346&quot;<br>SECURITY_PASSWORD_SALT = &quot;40113095-16f5-4e85-8140-98234802938320498&quot;</pre><p>The secret key and the password salt used above are just random strings. I use the python function `uuid.uuid4()` to generate these strings</p><p>The instance folder serves multiple purposes</p><ol><li>Being out of version control, it can safely contain secret keys</li><li>It can also be used to store the variables which need to change between different installations of the app. We will change all the above keys when the app is deployed to production. So a different instance/application.cfg.py will be used when we deploy.</li></ol><h3>Setting up the virtualenv and installing the requirements</h3><p>When working with python apps, it is always preferable to install the packages inside a virtual environment.</p><p>We can use <a href="https://virtualenvwrapper.readthedocs.io/en/latest/">virtualenvwrapper</a> to set up the python environment for running the app. Create a virtualenv like this</p><p>mkvirtualenv -p python3 flask_boilerplate</p><p>And with the virtualenv active, install the requirements using pip install -r requirements.txt. Here is the requirements.txt for this project</p><pre>flask<br>sqlalchemy<br>flask_sqlalchemy<br>flask_script<br>flask_migrate<br>flask_security<br>mysqlclient<br>bcrypt<br>ipython</pre><p>Refer the doc pages for <a href="https://virtualenv.pypa.io/en/latest/">virtualenv</a> and <a href="https://virtualenvwrapper.readthedocs.io/en/latest/">virtualenvwrapper</a> for more details if you are new to this.</p><p>With the environment set, we can start reviewing the server code.</p><h3>The main script — server.py</h3><p>The application server can be started by calling python server.py. Here is the code</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dde02979c79fb02c616c35e560964e47/href">https://medium.com/media/dde02979c79fb02c616c35e560964e47/href</a></iframe><p>That’s all. The app_factory.create_app function is the only entry point for our application. The entire architecture of the application can be understood by working backwards from here. Note that the application object created here, will also be used by us later when we run the app using uwsgi and serve it behind a nginx proxy. But more on that in the next post.</p><h3>The app_factory module</h3><p>This is the heart of the application. It is placed in app/app_factory.py</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/43de63e2633d0a4badcdc0bcdf833829/href">https://medium.com/media/43de63e2633d0a4badcdc0bcdf833829/href</a></iframe><p>This function will be invoked not just by server.py, but by any number of other scripts which need an application instance to work with. As we can see in the above example, the create_app function creates a flask app instance, sets some config variables on it, registers some blueprints, initializes some services and returns the ready to use app.</p><p>The flag instance_relative_config set to True in the first line allows us to read the application&#39;s config information from inside the instance folder. We initialize the config with the variables defined in app/default_config.py. This can include all the values which can be version committed. For secret keys, I prefer to initialize them to empty strings here, so that the app won&#39;t break even if the keys are absent in instance/application.cfg.py</p><p>The next line app.config.from_pyfile can load the python file with the specified name present in the instance folder.</p><p>Now with all the database credentials loaded from the config file, we then initialize the db handler, register the blueprints and initialize the Flask-Security extension which is used to provide support for user registration and login.</p><h3>The model layer</h3><p>For the purpose of this example, I have only defined the models required by FlaskSecurity for user authentication. Here is models/user.py</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/97bc559cd70b8a70524568aeefd6ae8a/href">https://medium.com/media/97bc559cd70b8a70524568aeefd6ae8a/href</a></iframe><p>This pattern is straightaway taken from the Flask-Security quickstart example. This library requires a Role model also to be specified for specifiying various roles for users like administrator, editor etc. We are not using roles in this example though. The user_datastore created here is then used in the create_app method when registering the Security module.</p><h3>The views</h3><p>As mentioned earlier, we are using the blueprints pattern to modularize the urls. In our app, we are classifying the urls into 2 categories — pages and api — defined in views/main_pages_bp.py and views/main_api_bp.py respectively. The former is supposed to render html responses while the latter will render json responses.</p><p>Here is views/main_pages_bp.py</p><pre>from flask import Blueprint, render_template</pre><pre>main_pages_bp = Blueprint(&#39;main_pages_bp&#39;, __name__)</pre><pre><a href="http://twitter.com/main_pages_bp">@main_pages_bp</a>.route(&quot;/&quot;)<br>def index():<br>    return render_template(&quot;index.html&quot;)</pre><pre><a href="http://twitter.com/main_pages_bp">@main_pages_bp</a>.route(&quot;/about&quot;)<br>def about():<br>    return render_template(&quot;about.html&quot;)</pre><p>And here is views/main_api_bp.py</p><pre>from flask import Blueprint, jsonify</pre><pre>main_api_bp = Blueprint(&#39;main_api_bp&#39;, __name__)</pre><pre><a href="http://twitter.com/main_api_bp">@main_api_bp</a>.route(&quot;/&quot;)<br>def index():<br>    return jsonify({<br>            &quot;status&quot;: &quot;success&quot;<br>        })</pre><h3>Templates</h3><p>The template files specified in the render_template method used in the views need to be present in the templates folder. We can create a standard layout which can be extended by each page</p><p>templates/layout.html</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/869cceb7a55e95788619a2ff0d2f71cc/href">https://medium.com/media/869cceb7a55e95788619a2ff0d2f71cc/href</a></iframe><p>The various concepts used in the above template — like import, block, macros, include etc are explained in detail in the <a href="https://jinja.palletsprojects.com/en/2.11.x/templates/">Jinja2 documentation</a> The include statements are basically partial templates which can be defined in separate files and loaded in other templates. The blocks are sections which can be overridden by any child template. Macros are similar to functions. The various partial templates defined in app/templates/layout_sections can be explored directly in the repository code. In macros.html, some utility functions have been defined like this</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a2766aa31bfec7ca166b37c72052312c/href">https://medium.com/media/a2766aa31bfec7ca166b37c72052312c/href</a></iframe><p>Now the template for home page — index.html is defined as follows</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08df517e636b8fa64f68e3f3d388e65a/href">https://medium.com/media/08df517e636b8fa64f68e3f3d388e65a/href</a></iframe><p>Thus we have an example of how to inherit from a layout and override specific blocks.</p><h3>Static folder</h3><p>In flask, the folder named as static is assumed to be the static folder meant to store and serve assets by default. The name of the folder can be customized if required, but there is no reason to do so usually. So here we have app/static folder with js and css files in it. These files were included in the templates using the macros.static_js_tag and macros.static_css_tag as shown above.</p><p>I have just placed some alert messages in the static code — just to illustrate that those files have been loaded. In further tutorials we will further explore how to bundle these static assets together while deploying to production.</p><h3>The migration script — dbmanager.py</h3><p>Now that we have reviewed the contents of the app folder, we are ready to run the application. But we need to create the database tables prior to that. While smaller flask-sqlalchemy projects would have just used the db.create_all method defined on the FlaskSQLAlchemy library for this, for a larger project we need a more scaleable method.</p><p>Typically, for large website projects, we need a database migration tool to manage the schema changes. The database schema will keep changing through the lifetime of the project — with constant addition, modification and deletion of various tables and constraints. Web frameworks handle this by using db migration tools — which have a chronological versioning mechanism. These tools will generate migration scripts, with each migration script having a reference to its immediate predecessor. Thus at any point in future, anyone who checks out the project, can just run the migrations scripts in order and get the database to the current schema. This is a very crucial requirement when multiple developers are working on the project.</p><p>In our case, we are going to use a library called flask_migrate to generate and manage the migrations. This is based on the alembic library which is the standardly used migration framework for SQLAlchemy.</p><p>The migration script looks like this</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fee1c89b493ae90825085d71d0585509/href">https://medium.com/media/fee1c89b493ae90825085d71d0585509/href</a></iframe><p>You can consider this as boilerplate code. We are creating a Manager instance using flask script - which is basically an interface for generating command line utilities for flask. We are then registering our app in the manager. You can see how the app_factory pattern was useful here. It let us generate an instance of the app for the migration tool to introspect the registered models. We then add a command called db which uses flask_migrate. The above code automatically ensures that the following commands are readily available</p><p>python dbmanager.py db init - To generate the migrations folder for the first time</p><p>python dbmanager.py db migrate - To generate the migration scripts whenever we make any changes in our models</p><p>python dbmanager.py db upgrade - To apply the generated migrations in the chronological order so that the database reaches the correct schema state</p><p>python dbmanager.py db downgrade - To downgrade the database schema to a previous version - thus rolling back the changes.</p><p>The migrations folder is auto-generated by running python dbmanager.py db init. Whether to version commit this folder or not is a choice for the developer to make. I prefer version committing it, so that others can get started by running just python dbmanager.py db upgrade</p><p>So let’s just run python dbmanager.py db upgrade. You will be able to see a message showing that the migration was applied successfully.</p><h3>Running the server</h3><p>Now that everything is ready, the development server can be initialized by running python server.py</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DchucpBN2WvbC7Sw.png" /></figure><p>You can register an user account by clicking on the “Register” link and signing up. Then you can login using those credentials. The login and register pages don’t have any styling as they use the default html templates defined by the flask_security library. In the next posts, we can see how to customize these templates to use our layout.</p><h3>Using cli.py</h3><p>The last remaining script left to review is cli.py</p><p>This is equivalent to the management shell offered by frameworks like Django. I have kept it very simple for the sake of this tutorial. But we can add a lot more features here as we will see in subsequent posts</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c8268cab6b4149ceb7050d2cf8773d58/href">https://medium.com/media/c8268cab6b4149ceb7050d2cf8773d58/href</a></iframe><p>To understand the reasoning behind second and third lines, you need to understand the concept of <a href="https://flask.palletsprojects.com/en/1.1.x/appcontext/">application contexts</a> and <a href="https://flask.palletsprojects.com/en/1.1.x/reqcontext/">request contexts</a> in flask and how and why a request context needs to be pushed in the shell as explained <a href="https://flask.palletsprojects.com/en/1.1.x/shell/">here</a></p><p>But these are pretty advanced topics. So for the sake of this tutorial, just assume it as boilerplate code.</p><p>We can use this tool by calling the script like this and fiddling around with various objects like this</p><pre>(myvenv) surya@devbox:~/personal/flask_pattern_minimal_with_auth$ python cli.py</pre><pre>In [1]: models.User.query.count()</pre><pre>Out[1]: 2</pre><pre>In [2]: models.User.query.all()</pre><pre>Out[2]: [<a href="/service/mailto:user1@gmail.com">user1@gmail.com</a>, <a href="/service/mailto:user2@example.com">user2@example.com</a>]</pre><h3>Conclusion</h3><p>In the first post of this series, we have built a very basic flask app using app_factory pattern. It does nothing except to let you register an user account and login. And it lays out the the skeleton of various concepts like templates, static folders etc which will be further explored in the subsequent posts.</p><p><em>Originally published at </em><a href="https://techonometrics.com/posts/basic-flask-app-factory-pattern/"><em>https://techonometrics.com</em></a><em> on April 15, 2020.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dbb891cdf69f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to setup basic web push notification functionality using a Flask backend]]></title>
            <link>https://suryasankar.medium.com/how-to-setup-basic-web-push-notification-functionality-using-a-flask-backend-1251a5413bbe?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/1251a5413bbe</guid>
            <category><![CDATA[progressive-web-app]]></category>
            <category><![CDATA[service-worker]]></category>
            <category><![CDATA[web-push-notifications]]></category>
            <category><![CDATA[sqlalchemy]]></category>
            <category><![CDATA[flask]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Sat, 08 Feb 2020 12:20:05 GMT</pubDate>
            <atom:updated>2020-03-10T09:23:08.948Z</atom:updated>
            <content:encoded><![CDATA[<p>Web browsers have become increasingly very powerful over the past few years. Many features which were earlier offered only by mobile apps are now available on web as well. This has primarily been due to Google’s investments in Chrome browser (and Mozilla’s work on Firefox as well) and some allied technologies like <a href="https://developers.google.com/web/progressive-web-apps">Progressive Web Apps</a></p><p>Service Workers are the core piece of the technology that enabled PWAs to compete with native apps in terms of the functionality offered. You can get a good introduction to how these service workers work from this <a href="https://developers.google.com/web/fundamentals/primers/service-workers">tutorial created by google</a> and this <a href="https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API">MDN article</a>. As mentioned in those articles, service workers can do a bunch of great things like</p><ol><li>Execute an action which needs to talk to the server on the background — thus removing the pesky loading spinners</li><li>Act as a network proxy — caching various assets required by the site and letting a basic site be loaded even without network connectivity</li><li>Receive push notifications from the server and display them to the user.</li></ol><p>In this post we will just stick to the last mentioned feature — Web Push Notifications. It is easy to add support for the same by integrating third party tools like <a href="https://webengage.com/">WebEngage</a>, <a href="https://onesignal.com/">OneSignal</a>, <a href="https://clevertap.com/">Clevertap</a> etc.</p><p>While these third party tools are sufficient for most purposes, if we intend to incorporate push notifications as a significant part of the user experience, it will be worthwhile to implement it ourselves. Google web fundamentals course has a detailed tutorial <a href="https://developers.google.com/web/fundamentals/push-notifications">here</a> on this topic. There is also a <a href="https://developers.google.com/web/fundamentals/codelabs/push-notifications/">codelabs project</a> which provides a demo project which can be cloned and experimented with. And then there is this <a href="https://web-push-book.gauntface.com/">Web Push Book</a> which is probably the most extensive tutorial available on the subject.</p><p>While these tutorials are quite detailed, there is also a need for a step by step tutorial that starts from scratch and progressively builds the features. This series intends to be that. In the first post of the series, we will just set up a bare minimal application with Web Push support just to demonstrate how to get it up and running. In the subsequent parts, we will improvise the user experience and add more features.</p><p>Before we proceed further, you should read through this <a href="https://web-push-book.gauntface.com/chapter-01/02-how-push-works/">post</a> to get an understanding of how the technology works. If you are short of time, here is a concise version -</p><ul><li>Javascript is single threaded. This means that the when a website is loaded, the main thread will always be occupied in rendering and handling user interactions. But this limits the user experience that we can provide to the users. There was a need for threads which can run in the background without disturbing the user interface.</li><li><a href="https://www.html5rocks.com/en/tutorials/workers/basics/">Web workers</a> were developed to satisfy this requirement.</li><li>Service workers are a type of web workers, via which we can instruct the browser to do certain specific background tasks.</li><li>One of the tasks that they can do is to listen for Push notifications sent by the server which can then be displayed to the user.</li><li>Chrome pioneered support for this feature. Firefox incorporates it as well now.</li></ul><p>So, with this basic understanding of WebPush, let’s dive into the code now.</p><h3>Our hero — The Service Worker</h3><p>Let’s first see what a service worker js file looks like. A side-note — I have taken the scripts used in the <a href="https://developers.google.com/web/fundamentals/codelabs/push-notifications/">Google Codelabs project</a> and modified them to keep only what is essential for a simple demo.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/847bbb1a945835eef2c56ef59bb06167/href">https://medium.com/media/847bbb1a945835eef2c56ef59bb06167/href</a></iframe><p>The code should be easy to understand for anyone with a basic javascript knowledge. The main unusual feature in the above code is the keyword self. It is a reference to the <a href="https://developer.mozilla.org/en-US/docs/Web/API/ServiceWorkerGlobalScope">ServiceWorkerGlobalScope</a> which is an interface representing the scope of the service worker. Since the service worker does not have a browsing context, this self reference provides the interface for various functionalities which are usually provided by the window object in the browsing context. Just like we would do window.addEventListener, here the event handlers of the service worker are defined using self.addEventListener</p><p>In the above code, we have listed 3 event handlers. The ones handling install and activate are just logging the events for our debugging purposes. The push event handler is also simple to understand. It reads the text data sent with the event, checks if it is a JSON format, and if so extracts a title and body from it. If it is not in JSON format, it just assumes that the entire string is to be used as the body, with title set as &quot;Untitled&quot;. As we will see below, this will help us debug the service worker from the client&#39;s chrome developer tools itself.</p><h3>Registering the service worker</h3><p>The above service worker needs to be registered by the website so that the browser knows about it. Here is the registration script which we are going to use — register_service_worker.js</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/38ef62fdc69e67f458784a97a8b7ed23/href">https://medium.com/media/38ef62fdc69e67f458784a97a8b7ed23/href</a></iframe><p>The last function — registerServiceWorker is the main function which will be invoked to register the service worker. It accepts 3 arguments</p><ol><li><strong>serviceWorkerUrl</strong> — The url which can be used by the browser to load the service worker. In our case, we will place the service worker js in the static folder of our flask app and the url will hence be /static/service_worker.js</li><li><strong>The application server public key</strong> — The second argument is the vapid public key. This is the public key component of a public key — private key pair which is used to securely authorize the communication between the push server and the push service running on the client. The mechanism is explained in more detail <a href="https://developers.google.com/web/fundamentals/push-notifications/subscribing-a-user#applicationserverkey_option">here</a>.</li><li><strong>apiEndpoint</strong> — Finally the third argument is the api end-point which will be used to save the push subscriptions generated by the clients. In our example we will define an endpoint /api/push-subscriptions which will do this job.</li></ol><p>The registerServiceWorker function calls the subscribeUser function which in turn calls the swRegistration.pushManager.subscribe method which generates the unique endpoint which will be used by the push server to send push notifications to the client. A side-effect of the .subscribe function call is that it will request permission from the user for showing push notifications, if the user has not yet allowed it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*axGdkpCd5l9MDYY0.png" /></figure><p>This is sufficient for the sake of this simple example. But a good user experience requires that we control when and how this request alert is displayed by the browser. That can indeed be done using the Notification.requestPermission() method described <a href="https://developers.google.com/web/fundamentals/push-notifications/subscribing-a-user#requesting_permission">here</a>. But for the sake of brevity, let&#39;s keep that out of this tutorial&#39;s scope as the primary aim of this first post in the series is just to get the push notifications working with the least amount of effort.</p><p>Now, the Push Subscription json generated by the swRegistration.pushManager.subscribe function call above looks like this</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/03ae67a3b2630e53ed8516cc72fbc0fc/href">https://medium.com/media/03ae67a3b2630e53ed8516cc72fbc0fc/href</a></iframe><p>This json should be stored so that the server can later send the desired push notification to this address. This is done by the updateSubscriptionOnServer function which posts the subscription json to the api endpoint which we will describe below.</p><h3>Structure of the app</h3><p>Now that we have reviewed both the service worker script and the script required to register it, we can start discussing the structure of our server which will be used to serve the html pages and the api endpoints.</p><p>We are going to build a simple Flask server as the backend and configure it to serve a simple website with a service worker file which can provide Web Push support. The code is available <a href="https://github.com/SuryaSankar/webpush_setup">in this repo</a>. The app being used in this first post of the series is under the folder named basic_functionality</p><p>The app is structured like this</p><pre><br>├── app.py<br>├── instance<br>│   └── application.cfg.py<br>├── requirements.txt<br>├── static<br>│   ├── register_service_worker.js<br>│   └── service_worker.js<br>├── templates<br>│   ├── admin.html<br>│   └── index.html<br>└── webpush_handler.py</pre><p>The instance folder will be missing if you clone the github repo. It is a folder which is not meant to be version committed as it will contain the secret keys (to be explained below)</p><p>Before we review each of these files one by one, let’s first see how to get the environment ready</p><h3>Setting up the virtualenv and installing the requirements</h3><p>We can use <a href="https://virtualenvwrapper.readthedocs.io/en/latest/">virtualenvwrapper</a> to set up the python environment for running the app. Create a virtualenv like this</p><p>mkvirtualenv -p python3 simple_webpush</p><p>And with the virtualenv active, install the requirements using pip install -r requirements.txt. Refer the doc pages for <a href="https://virtualenv.pypa.io/en/latest/">virtualenv</a> and <a href="https://virtualenvwrapper.readthedocs.io/en/latest/">virtualenvwrapper</a> for more details if you are new to this.</p><p>With the environment set, we can start reviewing the server code.</p><h3>Main server module — app.py</h3><p>The full code of app.py is available <a href="https://github.com/SuryaSankar/webpush_setup/blob/master/basic_functionality/app.py">here</a>. We are using a simple Flask app pattern for this example. If you are new to Flask, going through this <a href="https://flask.palletsprojects.com/en/1.1.x/quickstart/">quickstart tutorial</a> first will help. Note that this simple app structure is suitable only for development purposes. If we intend to deploy the application to production, we need to use a more scaleable architecture. But that is outside the scope of this first post.</p><p>Let’s review the code of app.py section by section now.</p><pre><br>from flask import Flask, render_template, request, jsonify<br>from flask_sqlalchemy import SQLAlchemy<br>from webpush_handler import trigger_push_notifications_for_subscriptions</pre><pre>app = Flask(__name__, instance_relative_config=True)<br>app.config.from_pyfile(‘application.cfg.py’)</pre><p>We are importing the essential flask and sqlalchemy modules. We are also importing a method which will be used for triggering the push notifications (to be discussed below)</p><p>The app is created using the flag instance_relative_config set to True. This lets us define some secret config information in a file inside a folder called instance which can be kept out of source control. The next line instructs the app to load the config from a file named application.cfg.py which is expected in the instance folder. So just create a folder named instance and create a file named application.cfg.py there. We will add the public and private keys required for sending web push notifications here.</p><p>Here is how the instance/application.cfg.py should look like</p><pre>VAPID_PUBLIC_KEY = &quot;BBBAASD#DFDFFFFFFFFFFFFFFFFFFFFFFF&quot; </pre><pre>VAPID_PRIVATE_KEY = &quot;yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy&quot; </pre><pre>VAPID_CLAIM_EMAIL = &quot;<a href="https://techonometrics.com/cdn-cgi/l/email-protection">[email protected]</a>&quot;</pre><p>To generate this public key-private key pair, you can use this website — <a href="https://web-push-codelab.glitch.me/">https://web-push-codelab.glitch.me/</a> Alternatively you can also use this npm command line utility — <a href="https://github.com/web-push-libs/web-push#command-line">web-push</a>.</p><p>The next section in app.py takes care of the database layer. The app needs a database connection to store the push subscription information generated by the browser. With Flask we generally use SQLAlchemy as the database ORM layer and FlaskSQLAlchemy as the linker library. You can refer their documentations <a href="https://docs.sqlalchemy.org/en/13/">here</a> and <a href="https://flask-sqlalchemy.palletsprojects.com/en/2.x/quickstart/">here</a> if you are unfamiliar.</p><pre>app.config[&#39;SQLALCHEMY_DATABASE_URI&#39;] = &quot;sqlite://&quot; </pre><pre>db = SQLAlchemy(app) </pre><pre>class PushSubscription(db.Model):    <br>  id = db.Column(db.Integer, primary_key=True, unique=True)<br>  subscription_json = db.Column(db.Text, nullable=False)</pre><pre>db.create_all()</pre><p>In the above section, we have done the following</p><ol><li>The database connection string sqlite:// instructs SQLAlchemy to use an in-memory sqlite database. Since it is in-memory it is non-persistent and the information will be lost if the server process is shut down. But this is sufficient for the purposes of this simple demo.</li><li>In the next line we have initialized the db handle and then we have defined a model named PushSubscription which will be used to store the subscription information. We will review the structure of this json shortly.</li><li>The next line db.create_all() takes care of creating the database tables - just one table in this case named push_subscription</li></ol><p>With the database configured, we can now proceed to register the required url endpoints. The first route home_page renders the template index.html.</p><pre><a href="http://twitter.com/app">@app</a>.route(&quot;/&quot;)<br>def home_page():<br>    return render_template(&quot;index.html&quot;)</pre><p>The index.html rendered above can just be a bare minimum template which loads the register_service_worker.js script which we had described earlier and then invokes the registerServiceWorker function defined in the script. As required by Flask, we will place this in the `templates` folder.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e56d87e7a16e79d217d423413c0b9d1d/href">https://medium.com/media/e56d87e7a16e79d217d423413c0b9d1d/href</a></iframe><p>We have placed both the js files in the static folder and loaded the register_service_worker.js script directly in the html. That in turn takes care of loading the service worker using the url we have passed as the first argument. The second argument is supposed to be the public key as we discussed earlier when reviewing the registration script. Instead of hardcoding it, we are referring to it using a config variable. Flask will take care of loading this variable from the value we placed in instance/application.cfg.py earlier. And the third argument is supposed to be the api endpoint to which the registration script will upload the subscription_json. We will define it as follows in app.py</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/692e068603bdbd5cc47f783c0f0ca5a4/href">https://medium.com/media/692e068603bdbd5cc47f783c0f0ca5a4/href</a></iframe><p>As it can be seen, the above api endpoint first checks if there is already a matching PushSubscription object with the same subscription_json, and if so returns it. So when we reload the page, even when the browser resends the same subscription_json due to the registerServiceWorker function being invoked again, it won&#39;t cause any error on the server. If there is no matching subscription_json on the other hand, a new entry is created and returned to the browser.</p><p>The setup done so far is enough to check that the service worker works as expected. Chrome developer tools has tools which let us test the service workers from the client itself. So let’s do that first before seeing how to send the push notifications from the server.</p><h3>Testing the client side functionality</h3><p>We first start the server by running the flask run command</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fHKkts4P8hPj8lRA.png" /></figure><p>Once the server starts, open localhost:5000 in the browser. Keep the Developer tools open on the side. In case you haven’t done this before, it is available under “More Tools” section in Chrome menu. Or you can press Ctrl + Shift + I. Once the site loads, you should see a dialog which asks you for notification permissions. Click on Allow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*60AhzYKxFCVMywpI.png" /></figure><p>When clicking on “Allow”, if we inspect the Network tab in Developer tools, we can see a POST request being sent to the API endpoint.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*b6QTbjZ5jiWU1cxa.png" /></figure><p>In the response we can see the subscription details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KT6MExD4OpEJj4qT.png" /></figure><p>Now that the registration of service worker is done, let’s test out its notification handler code. In the developer tools, click on the Application tab and click on the Application -&gt; ServiceWorkers section in the side menu.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*N14TS56o4n5b07AC.png" /></figure><p>It shows the registered service worker on the localhost site. The status should show as activated and running. We can now test that this service worker is working as expected on the client side using this devtools interface itself. Type some string in the text-box labeled Push and click on the Push button. You should immediately see a push notification popup on the side. The title shows as “Untitled” because that is how we have coded (Check the service_worker.js code above)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1Ztl__eCXEgP9Jda.png" /></figure><p>We can also test how the service_worker will show the notification if we send it in the json format that we have coded the service worker to handle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cY-wQC0iJ_nRXiGW.png" /></figure><p>Thus we have verified that the notification handler in the service worker is working as expected. The next step is to go through the server setup to send the push notifications</p><h3>Server code to send web push notifications</h3><p>We will set up a simple admin page to send push notifications. A proper admin dashboard requires user authentication and authorization. But we will leave that for the next posts in this series. Here we will just focus on the code essential for sending out the notifications.</p><p>We first define the admin page like this in app.py</p><pre><a href="http://twitter.com/app">@app</a>.route(&quot;/admin&quot;)<br>def admin_page():<br>    return render_template(&quot;admin.html&quot;)</pre><p>The html is defined as follows with a simple form which allows you to enter a title and body of the push message and submit it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/995e370ea6d7974731b45cf66c1d0631/href">https://medium.com/media/995e370ea6d7974731b45cf66c1d0631/href</a></iframe><p>On submission, it hits an api endpoint which is defined as follows. This endpoint uses the trigger_push_notifications_for_subscriptions function to send the notification to all the subscribers (only one in our case)</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f1ba03c1247e8b4c9998473a0da392d6/href">https://medium.com/media/f1ba03c1247e8b4c9998473a0da392d6/href</a></iframe><p>The trigger_push_notifications_for_subscriptions is defined in webpush_handler.py which we had already imported in app.py</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aa84a6c2b11ab01cd9d47e1908f23a61/href">https://medium.com/media/aa84a6c2b11ab01cd9d47e1908f23a61/href</a></iframe><p>It’s a straightforward invocation of the function defined by the pywebpush library which we had already installed as part of the requirements. We are signing it with the vapid private key which we had copied to the instance/application.cfg.py. Since the client already knows the public key, it will be able to validate that the push message is coming from the authorized server only.</p><h3>Testing out the admin form for sending push notifications</h3><p>To test this functionality, let’s visit localhost:5000/admin Here, if we fill the title and body and submit, we can see a push notification like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZMnBnicb1UGdAiBR.png" /></figure><p>If you are seeing a notification like above, congratulations you have successfully built a simple website with a very basic web-push notification support.</p><h3>Conclusion</h3><p>In the above tutorial we have reviewed how to build a simple app with support for push notifications. We can continue building features on top of this as follows</p><ol><li>Improving the push notification handler on client side — by implementing the Notification.requestPermission() method, by adding more options for controlling the appearance and behavior of the notification, adding support for images etc</li><li>Modifying the app structure to make it more scaleable and ready to deploy</li><li>Use a persistent database for storing the push subscriptions</li><li>Implement user authentication and authorization so that the admin form can be opened only for certain users.</li><li>Implementing support for user segmentation on the backend, so that we can send the notifications to specific target segments</li></ol><p>These features will all be done in the subsequent posts of this series.</p><p><em>Originally published at </em><a href="https://techonometrics.com/posts/web-push-notifications-basic-functionality-using-flask-backend/"><em>https://techonometrics.com</em></a><em> on February 8, 2020.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1251a5413bbe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An introduction to the Jupyter ecosystem]]></title>
            <link>https://medium.com/techonometrics/an-introduction-to-the-jupyter-ecosystem-72005f7bdd78?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/72005f7bdd78</guid>
            <category><![CDATA[jupyter-notebook]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[jupyterhub]]></category>
            <category><![CDATA[jupyter]]></category>
            <category><![CDATA[jupyterlab]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Sat, 05 Oct 2019 01:20:05 GMT</pubDate>
            <atom:updated>2019-10-24T21:18:19.431Z</atom:updated>
            <content:encoded><![CDATA[<p>I had recently setup a Jupyterhub server for my company and wanted to blog about the same. Before we get into the setup instructions, it might be useful to have a brief overview of the Jupyter eco-system. When I started I was confused by the various terms which were being thrown around. So this post will be kind of a glossary of the various technologies which together constitute the Jupyter eco-system.</p><h3>First there was the shell</h3><p>According to multiple surveys, Python is either the <a href="https://pypl.github.io/PYPL.html">most popular</a> or the <a href="https://insights.stackoverflow.com/survey/2019#most-popular-technologies">second most popular</a> programming language in the world. Frameworks like Django, Flask etc have made it a major contender in the server side application development landscape where it is second only to Javascript in the adoption rate. But the biggest success story of Python has been as the language of scientific computing and data processing. Python has become the go-to language for professionals from other domains for whom programming is only a part of their job description and would hence like a language which just gets the job done and gets out of the way. Python fit this requirement very well. The main reason for this was python’s readability, gentle learning curve and the community’s general preference for explicit code over implicit magic.</p><p>But there was another major factor as well. Python being an interpreted language, was one of the first major languages to offer a powerful interactive shell. The availability of an interactive shell is particularly important during the research and prototyping stage when you need to play around with data and see immediate results. Python shell allowed you to execute code blocks one at a time and view the results. Over years the need was felt for a richer and more interactive python shell and thus <a href="https://ipython.org/">IPython shell</a> was born.</p><p>Ipython shell offered many advantages over the built in python shell like tab completion, inspection of object internals, access to the underlying bash shell etc. You can check it out yourself by doing pip install ipython and then invoking ipython command. You will see the differences with the basic python shell.</p><p>The next major development that happened was Ipython notebooks. These were basically browser based ipython shells which could communicate with a python process running on the local machine or a remote server. Plus, the whole interactive session could be saved as “.ipynb” files which saved the inputs and outputs of all the cells in a json format, allowing the notebooks to be shared and re-executed by others. These ipython notebooks became very popular in the research community and in 2014, the notebook part of the ipython project was separated from the command line client and was spun off as a separate project — <a href="https://jupyter.org/">Jupyter</a></p><h3>Uses of Jupyter notebooks</h3><p>If you are a research scientist or data analyst who is starting work on a new topic, one of the first things you are going to do is to open an ipython console and play around with the data you have. Once you are sure that you have come up with something that is worth sharing with your peers, you will want to convert the exploratory analysis you just did into a presentable format. Jupyter notebooks were developed to make this process very simple by just making the python console itself a shareable document.</p><p>Here are some examples,</p><ol><li><a href="https://nbviewer.jupyter.org/github/maayanlab/Zika-RNAseq-Pipeline/blob/master/Zika.ipynb">A scientific paper on RNA sequence analysis published as a jupyter notebook</a></li><li><a href="https://nbviewer.jupyter.org/github/jckantor/CBE20255/blob/master/notebooks/02.01-Balancing-Reactions.ipynb">A notebook which has been created to explain Stoichometry for Chemical Engineering students</a></li></ol><p>As you can see from above examples, the notebook is basically a collection of “Cells”, a cell being a block of code / text. Text and code blocks can be interspersed freely, so if you are exploring a mathematical formula for example, you can freely mix the textual annotations which explain the formula and pythonic code which can demonstrate the formula in action by passing some inputs to a method and retrieving the output. And the fact that these notebooks are built on top of web technologies, means that the jupyter notebook consoles are capable of supporting a much richer output than plain shell based consoles. Most visualization libraries these days provide out of the box support to render the output on a jupyter notebook. Here is an <a href="https://anaconda.org/jbednar/census/notebook">example notebook analysing US census data</a> with some rich visualizations.</p><p>Researchers are not the only ones who can take advantage of the capabilities of Jupyter notebooks. Content writers can benefit as well. So a financial journalist reporting on the financials of a company can easily insert small snippets of code which pull the latest financial data of the company available on the internet and run some analysis on it and convert it into attractive visualizations. Similarly one can imagine how so many other kinds of content writers would benefit — technical writers writing posts on scientific topics which can be annotated with visualizations, journalists who want to report on some political or economic stats, the list is endless.</p><h3>How to install and use Jupyter notebooks</h3><p>The easiest way to get started with Jupyter is to <a href="https://docs.anaconda.com/anaconda/install/">install the Anaconda environment</a> on your local machine. It comes with Jupyter notebook bundled with it. Alternatively if you prefer pip as the package manager, you just install the python package as pip install jupyter and then you are ready to go. Calling jupyter notebook from the shell then spawns a server which can be interacted with via the web client.</p><h3>Nbconvert, Nbviewer</h3><p>Since Jupyter notebooks have become a great hit with researchers and content creators, they also have to be convertable into various other presentation formats. So there is a command line tool available for the same — <a href="https://nbconvert.readthedocs.io/en/latest/">nbconvert</a> using which Jupyter notebooks can be readily converted into a variety of presentation formats like HTML, PDF, Latex etc. The conversion is done simply by invoking the tool like this jupyter nbconvert --to pdf notebook.ipynb.</p><p>There is also a ready to deploy web server available called <a href="https://github.com/jupyter/nbviewer">Nbviewer</a>, which can be used to provide this Nbconvert as a service. So you can deploy Nbviewer on a server and allow users to submit links to jupyter notebooks and convert and render the generated html pages. There is a community maintained instance of nbviewer hosted <a href="https://nbviewer.jupyter.org/">here</a></p><h3>Integration with Blogging Engines/Static site generators</h3><p>Converting notebooks into html is particularly interesting because this means that the notebooks can be considered as just another kind of a text format like rtf or markdown for writing blog posts, with the added advantage of supporting various dynamic functionalities like visualizations. When the notebooks are exported, these visualizations become a part of the exported html/pdf document.</p><p>Since nbconvert allows you to convert notebooks to html/markdown formats, you can use any major static site generator like Jekyll or Hugo, by just including the nbconvert command in your build process. There are tutorials available for the same like <a href="https://www.linode.com/docs/applications/project-management/jupyter-notebook-on-jekyll/">here</a> and <a href="https://github.com/vlunot/nb2hugo/">here</a></p><p>Another option which has arrived on the scene recently is <a href="https://getnikola.com/getting-started.html">Nikola</a> . Nikola is a static site generator which includes support for ipython notebooks out of the box. If you are certain that most of your blogs will have some element of data analysis in them, using Nikola would be the straight forward option.</p><h3>JupyterLab</h3><p>The default web interface that you see after installing Jupyter is the classic jupyter interface. But there is a next generation UI — which is called <a href="https://jupyterlab.readthedocs.io/en/stable/">JupyterLab</a>. It provides several features which were not available in the classic interface like built in support for viewing markdown documents, a launcher which can be used to launch new notebooks or a text document or code consoles. And Jupyterlab also has an excellent eco-system of <a href="https://github.com/mauhai/awesome-jupyterlab">third party extensions</a>. In the next post, when we set up Jupyterhub, we will also set up the git extension for example.</p><h3>What is a JupyterHub?</h3><p>A Jupyterhub is basically a web server which allows jupyter notebooks to be created and run by various users on a remote machine which can then be accessed via JupyterLab interfaces on the browser. Basically the browser console is pointed towards the remote Jupyterhub server instead of the local Jupyter server as would have been the case if you had just invoked jupyter or jupyter lab on the command line.</p><p>If you are using jupyter notebooks only for your own consumption, then the local server is sufficient for your purposes. But if you have to host the notebooks on a remote server and provide access to others, then it becomes necessary to setup a JupyterHub. The server which is spawned when launching a Jupyter notebook, does not support authentication. JupyterHub does. You can setup user accounts and allow access only to authorized users. This is essential if you are going to host the notebooks on a server Jupyterhub is typically used by data teams in companies and universities. But even an individual content creator/data analyst can benefit from setting up a Jupyterhub.</p><h3>Why Jupyterhub can serve as a good content management system for a technical blog</h3><p>The Jupyter ecosystem has more than adequate tools necessary for a good content management system</p><ol><li>The Jupyterlab interface which has a decent Markdown editor with previewing ability is good enough for writing textual posts</li><li>As already discussed above, the notebook format is excellent for technical literature, as it can let the author create original analyses with visualizations. The fact that it is in code means that the analyses and the visualizations can be updated easily. By contrast, if you use a traditional content management system like wordpress, you might have to modify the visualizations via some other third party tool and re-insert it in the post. By making this process painless, jupyter notebooks encourage multiple iterations on technical content, just like a programmer would write a program.</li><li>As discussed above, the notebooks can be easily converted into html files. And multiple static site generators like Jekyll, Hugo, Pelican, Nikola etc have tools which allow you to do this conversion as a part of the site building process itself. So we can easily serve the generated content via one of these tools. (We will discuss this in more detail in a later post)</li><li>Jupyterlab has an extension which provides a GUI for git integration. So, you can keep your content in a git repository. For a technical blog, having git as the version control system is a natural choice. Thus the data is stored separate from the content management system and the server. Even if the server were to crash unexpectedly, the data would still be safe in a git repository.</li><li>Jupyterhub supports various <a href="https://github.com/jupyterhub/jupyterhub/wiki/Authenticators">authentication mechanisms</a>. Setting up a Google oauth authentication is very easy. So you can easily integrate it inside your company’s authentication system.</li><li>Python has several excellent visualization libraries like matplotlib, <a href="https://github.com/plotly">plotly</a>, <a href="https://bokeh.pydata.org/en/latest/index.html">bokeh</a> etc. Writing content as a jupyter notebook helps the author take advantage of these while creating visually appealing posts.</li></ol><h3>How to set up Jupyterhub server</h3><p>The simplest way to get a jupyterhub server up and running is to follow the instructions here — <a href="https://tljh.jupyter.org/en/latest/index.html">The Littlest Jupyterhub</a>. The installation script provided there takes care of the entire setup process</p><p><em>Originally published at </em><a href="https://techonometrics.com/posts/an-introduction-to-jupyter-ecosystem/"><em>https://techonometrics.com</em></a><em> on October 5, 2019.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72005f7bdd78" width="1" height="1" alt=""><hr><p><a href="https://medium.com/techonometrics/an-introduction-to-the-jupyter-ecosystem-72005f7bdd78">An introduction to the Jupyter ecosystem</a> was originally published in <a href="https://medium.com/techonometrics">Techonometrics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Waymo’s threat to Uber’s raison d’etre.]]></title>
            <link>https://suryasankar.medium.com/waymos-threat-to-uber-s-raison-d-etre-f605e53b7245?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/f605e53b7245</guid>
            <category><![CDATA[waymo]]></category>
            <category><![CDATA[uber]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Wed, 01 May 2019 12:20:05 GMT</pubDate>
            <atom:updated>2019-05-12T08:10:14.853Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*C-9R3g6xQMCxrgfJ2omXrg.png" /></figure><p>Google’s Waymo is a threat to Uber, but not just because it has the obvious head start in the autonomous car business. While that would indeed reduce Uber’s eventual market share in this segment, Waymo’s arrival also threatens Uber in a more fundamental way. In my opinion, it also threatens Uber’s ability to continue to operate the normal taxi business which is the far bigger business still. Here is why.</p><h3>Self driving cabs — Taxi business or Car rental business?</h3><p>The popular perception is that if the legislation permits fully autonomous vehicles eventually, self driving cars will have a major advantage over human driven cabs since they would have removed the labor cost entirely from the equation.</p><p>But more and more one thinks about it, using self driving cars for mass transportation looks lot more like a car rental business than like a taxi business. Instead of renting a car to drive myself, I am hopping on a car which drives itself. The unit economics will still be similar to the car rental business. Since there is no driver who will act on behalf of the company, the business will have to consider the aspect of rogue passengers who might damage or attempt to steal the vehicle by overriding it, assuming that the self-driving cars will still have manual overrides — both because of regulations, as well as because of the fact that passengers are highly unlikely to board a vehicle where they have absolutely no control if things go wrong.</p><p>And once we realize that we are talking about a modified version of car rentals, rather than a modified taxi business, we can also question the hypothesis that self driving cars will eventually replace taxis entirely in the near future. There are simply way too many obvious cultural and legal hurdles. Taxis can be hailed by everyone — even those who cannot drive. But it is highly doubtful if people who can’t drive will comfortably step inside a fully autonomous vehicle ( Even regulations might prohibit that. ). Can it happen 30 years later? Maybe. But it makes no sense to talk about what can happen 30 years later. That is simply far past the average lifetime of companies. Uber might have been long gone by then.</p><h3>The autonomous cab business or autonomous car software — which is the real play here?</h3><p>The problem discussed above is true even for first world countries — where traffic is highly regulated and predictable and law and order system can be expected to handle cases of damages to cabs properly. Then, it becomes a hopeless fantasy to ever imagine this business having any impact in emerging economies where traffic is much much higher and law and order enforcement is only now beginning to be taken seriously. The sensible conclusion is that this business will remain confined within a niche market for the next 10–15 years — limited to select cities in the US and Europe. In fact it is certain that Alphabet and Waymo also know this already. They are not entering this business to compete with Uber. They will be doing this in order to collect valuable data and perfect their algorithms. This will be a huge opportunity to understand people’s behaviour and help them progressively refine the user experience of their self-driving technology. In fact, it makes more sense for them to consider Tesla as a competitor. Tesla gets valuable data by bundling its self-driving technology with its cars and treating its customers as test subjects. Waymo will get to do this by launching this self-driving car taxi business and treating its customers as test subjects.</p><p>The technology will eventually be worth billions to automobile companies around the world. Waymo can become the monopoly software company which provides the software modules which will power the self-driving cars of all the automobile brands. It won’t be a surprise if even Tesla migrates to it at some point.</p><h3>The real threat to Uber</h3><p>Uber’s core business of human driven cabs will barely be affected by self driving cars for the next decade at least. And Uber is insisting that it wants to continue expanding in emerging economies like India — where self driving cars won’t be a factor for maybe even two more decades until the infrastructure can catch up to support it. Despite these, Waymo will still be a huge threat to Uber even in the short to medium term. Why?</p><p><strong>Because of the way Uber funds it business.</strong></p><p>Uber has so far not yet figured out how to make profits in this cut-throat cab business. Its dreams of being the one monopolistic platform where all rides are booked is in shatters now with strong competitors in every market — Lyft, Didi, Ola, Grab etc. It means there is no realistic timeline at which Uber will eventually be able to start charging users at rates where it can start covering its expenses. It will keep having to keep the prices down artificially waiting for all its competitors to die out. But then again, they are also heavily funded and are doing the same waiting game.</p><p>So the only way for Uber to survive has been to keep raising bigger and bigger rounds. They have done this so far by promising bigger and bigger market opportunities to the investors — Flying cars, Food delivery etc etc. The logic was that just like Amazon built a hugely successful AWS business by leveraging the infrastructure it had already built, Uber will also be able to leverage its huge network of riders and drivers to build allied businesses. But these side projects have not really shown any big traction so far. If we start analysing why, that will take another big post, so let’s leave that for another day.</p><p>Since these moonshot ideas have not materialized so far, Uber is increasingly under pressure to show that its core business can be profitable in itself without needing some allied businesses to rescue it.</p><p>But we already saw that there is no way Uber can increase the ride prices significantly — since they have strong competitors everywhere. This means they have to cut costs. One way to do this is to squeeze the drivers. Transfer as much costs on to them as possible and give them as little pay as possible. Uber is certainly trying this. But it is also encountering stiff opposition from drivers everywhere. In fact trade unionism which had been dead for several decades is experiencing a revival in many cities thanks to Uber.</p><p>So that bring Uber to its final trick — Tell investors that it will bring in self-driving technology for its cars eventually and remove the pesky drivers from the equation — thereby reducing the costs and making profits finally.</p><p>As long as self-driving car technology remains an unproven or limitedly proven technology, investors can keep taking comfort in the religious belief that Uber will eventually master it and keep dumping money into Uber hoping that they will invest it in research and implement the technology soon.</p><p><strong>Waymo debuting it much ahead of Uber causes 2 problems to this logic</strong></p><ol><li>Like others have mentioned, it proves that Waymo is a few years ahead of Uber in this respect. And artifiical intelligence is one field which benefits a lot from data. First movers do indeed have a big advantage here. And unlike other softwares, second movers cannot simply copy the features and catch up. Without the data, the system will simply be inferior. This means that it doesn’t make sense for investors to fund self-driving research in Uber if Waymo is anyway years ahead. Better to ask Uber to wait and eventually buy the software from Waymo when it starts selling that software as a commodity. So one less bullet point in Uber’s pitch deck when it tells investors the reason it needs more money for</li><li>But an even bigger problem will arise if and when Waymo’s taxi service shows the limitations of using self-driving cars as taxis. As discussed at the start of this post, it is quite likely that it will show itself to be a very niche business not really capable of showing a global scale. All the problems which the investors might not have either ignored or wrongly assumed as solvable, will now be glaringly obvious as unsolvable. It will be clear that even if those problems are solved by Waymo, it was at a great cost and solved only because Waymo’s (and Google’s) primary goal was in perfecting the software to be sold, and not in making self-driving taxis a great business in itself. In short, Waymo might end up proving that while the software for self-driving cars is a tremendously profitable business, using the software for running them as taxis might not be as attractive as it is dreamt to be. That will greatly dampen the story that Uber is selling.</li></ol><p>So the threat that Waymo poses to Uber is that it spoils the story that Uber has been selling its investors. And for a business which is yet to make profit, that is a pretty huge threat.</p><p><em>Originally published at </em><a href="https://techonometrics.com/posts/waymos-threat-to-ubers-raison-d-etre/"><em>https://techonometrics.com</em></a><em> on May 1, 2019.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f605e53b7245" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to access environment variables in a notebook on a Jupyterhub server]]></title>
            <link>https://medium.com/techonometrics/how-to-access-environment-variables-in-a-notebook-on-a-jupyterhub-server-c9f53e256bc9?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9f53e256bc9</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[jupyterhub]]></category>
            <category><![CDATA[jupyter-notebook]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Sun, 24 Feb 2019 12:20:05 GMT</pubDate>
            <atom:updated>2019-02-25T08:31:36.524Z</atom:updated>
            <content:encoded><![CDATA[<p>Recently I had to create some jupyter notebooks to analyse a database. Since the database credentials were not meant to be committed as part of the code, I decided to keep them as environment variables. And it turned out to be surprisingly tricky to access the environment variables in the notebook. The first complication was that it was a Jupyterhub server and not a single instance notebook. When you launch a jupyter notebook directly from the command line, it has access to the environment variables defined in the .bashrc startup scripts. But that is not the case here. We will typically run Jupyterhub as a daemon and the Environment variables need to be passed in via that daemon script. And then the config file we use for the hub must also know to load those variables when we launch a new notebook. After much trial and error and googling, the following steps worked.</p><h3>Step 1 — Add the environment variables to the daemon scripts</h3><p>Your jupyterhub startup script will typically reside at this place — /lib/systemd/system/jupyterhub.service</p><p>It should look like this</p><pre>[Unit] <br>Description=Jupyterhub </pre><pre>[Service] <br>User=root <br>ExecStart=/usr/local/bin/jupyterhub -f /home/username/jupyterhub_repo/jupyterhub_config.py </pre><pre>[Install] <br>WantedBy=multi-user.target</pre><p>Lets now add the desired environment variable in this file</p><pre>[Unit] <br>Description=Jupyterhub</pre><pre>[Service] <br>User=root<br>Environment=GOOGLE_API_KEY=&#39;somesecretkeystring&#39; Environment=GITHUB_API_KEY=&#39;anothersecretkeystring&#39;<br>ExecStart=/usr/local/bin/jupyterhub -f /home/username/jupyterhub_repo/jupyterhub_config.py</pre><pre>[Install] <br>WantedBy=multi-user.target</pre><p>Note that we are adding 2 directives The first directive Environment is used to initialize an environment variable. We can initialize as many variables as we want in the above manner. But this step in itself is not enough. You will find that you still cannot access the variable inside your jupyterhub notebook shells. The reason is that the above directive only passes the environment variables to the command mentioned in the ExecStart directive. You need to further tell the systemctl daemon to pass the variables to the created process as well.</p><h3>Step 2 — Passing the variables to the Jupyterhub process</h3><p>We use the PassEnvironment directive to achieve this</p><pre>[Unit] <br>Description=Jupyterhub</pre><pre>[Service] <br>User=root<br>Environment=GOOGLE_API_KEY=&#39;somesecretkeystring&#39; Environment=GITHUB_API_KEY=&#39;anothersecretkeystring&#39;<br>PassEnvironment=GOOGLE_API_KEY GITHUB_API_KEY<br>ExecStart=/usr/local/bin/jupyterhub -f /home/username/jupyterhub_repo/jupyterhub_config.py</pre><pre>[Install] <br>WantedBy=multi-user.target</pre><p>Now the variables have been passed to the Jupyterhub process. But there is still one more step left</p><h3>Step 3 — Passing the variables to the Jupyterhub process</h3><p>While the above step passed the variables to the Jupyterhub process, you will find that these are not yet available in your Ipython shell launched inside Jupyterhub. This needs to be done by making Jupyterhub pass these variables down when it spawns a new shell. We do this by modifying the jupyterhub_config.py that we use to configure our Jupyterhub instance. Note that the location of this file is already mentioned in the ExecStart command in the daemon script. You would usually place this file in the root of your jupyterhub repository</p><p>To pass the variables, add the following lines to your config file</p><pre>import os<br>for var in os.environ:<br>  c.Spawner.env_keep.append(var)</pre><p>This passes on all environment variables to the spawned shell. Alternatively, you can also choose to pass only specific variables in the above script.</p><p>With these steps, you can now start using the environment variables inside jupyterhub. There are some obvious limitations to this approach. We are being forced to define the environment variables inside the daemon script — which means we might have to duplicate the values which were already defined in the .bashrc startup script. But I think the main advantage is that environment variables are easily usable by everyone and when your notebook uses code like os.environ[&quot;GOOGLE_API_KEY&quot;], it makes it easy for other checking out the notebook to directly use it for their purpose by just changing the environment variables in their environment.</p><p><em>Originally published at </em><a href="https://techonometrics.com/posts/how-to-access-environment-variables-in-jupyter-notebook-on-a-jupyterhub-server/"><em>techonometrics.com</em></a><em> on February 24, 2019.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9f53e256bc9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/techonometrics/how-to-access-environment-variables-in-a-notebook-on-a-jupyterhub-server-c9f53e256bc9">How to access environment variables in a notebook on a Jupyterhub server</a> was originally published in <a href="https://medium.com/techonometrics">Techonometrics</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Great men finish last]]></title>
            <link>https://suryasankar.medium.com/great-men-finish-last-da80e737671c?source=rss-aad6b702d70a------2</link>
            <guid isPermaLink="false">https://medium.com/p/da80e737671c</guid>
            <category><![CDATA[trump]]></category>
            <category><![CDATA[xi-jinping]]></category>
            <category><![CDATA[india]]></category>
            <category><![CDATA[china]]></category>
            <category><![CDATA[history]]></category>
            <dc:creator><![CDATA[Surya Sankar]]></dc:creator>
            <pubDate>Mon, 26 Feb 2018 04:36:19 GMT</pubDate>
            <atom:updated>2018-02-26T04:36:19.104Z</atom:updated>
            <content:encoded><![CDATA[<p>So, China is moving to abolish term limits and essentially let Xi Jinping do a Mao if he prefers so.</p><p><a href="https://www.nytimes.com/2018/02/25/world/asia/china-xi-jinping.html">https://www.nytimes.com/2018/02/25/world/asia/china-xi-jinping.html</a></p><p>On the one hand, this is sad and scary because I had really wanted to believe that CPC had become a very pragmatic and stable political organization which would guarantee the stability and predictability that the global economy desperately needs now.</p><p>On the other hand, as an Indian, this is quite exciting because, it is yet another opportunity for India to rise to the top. America committed political suicide last year and now it looks like China is on its way too. People might disagree, but empirical evidence shows that concentrating power in the hands of one person always ends badly, no matter how patriotic or honest that person professes to be.</p><p>All nations are obsessed with great men. We vote for men who would promise to turn around huge nations by eliminating everyone who is corrupt, which usually means everyone who is not singing their praise. We praise entrepreneurs who hog all credit for themselves without any hint of the teams which actually got them there.</p><p>But history, since the time when Caesar destroyed Rome has had the same lesson. Great men leave a trail of destruction behind. Progress has happened only under smaller, humbler leaders who were willing to consult and co-operate with others.</p><p>Can anyone remember the prime ministers under whom England rose to become a global power? Not really. But we all remember Churchill as a great man. And that great man presided over the destruction of a 200 year old empire.</p><p>America’s rise to prominence in the early 20th century happened under a string of forgettable presidents — Howard Taft, Woodrow Wilson, Calvin Coolidge — none of them are names that anyone except the most hardened history buffs would remember.</p><p>Soviet Union’s greatest successes — the glory years of the space race, happened during the time of Kurushchev and Brezhnev — who were so forgettable that there are tons of jokes about how unremarkable and silly they were.</p><p>Mankind’s greatest achievement — Apollo 11, happened due to NASA’s collective effort. None of us know who were the brains behind the effort. We just know that the entire organization of NASA with hundreds and thousands of men and women achieved the impossible.</p><p>Closer home, it was the eminently forgettable and soft spoken Narasimha Rao, who managed to turn around India. India’s best years since independence have all been under soft spoken leaders who kept a low profile — Rao, Manmohan Singh, Vajpayee — none of them portrayed themselves as great men. The worst stagnation happened under Indira, who had portrayed herself as a Great leader.</p><p>China’s rise to prosperity happened under Jiang Zemin and Hu Jintao — again two very forgettable men who quietly compromised and negotiated the way for China to rise to the top. They did it so quietly that no one even noticed it happening.</p><p>But the public rarely appreciates such humility. We mistake humility for weakness. And then we bend over backwards before an emperor. We believe that Modi will create a new India. That Trump would make America great again. That Xi Jinping would show the world what China can do.</p><p>And that always ends badly.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da80e737671c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>