• Introduction
• Analysis
• Improvements
• Generalization
• Conclusion

Introduction

In this blog, we will take a look at the analysis of the Airbnb dataset for Boston and take a look at what story the dataset will tell us. We will do some exploratory data analysis, and some data cleaning, and answer some questions involving

1. Top and bottom listings,
2. Reviewers and their rankings,
3. Reviewer’s general mood to check review reliability,
4. The mood around listings, customer satisfaction, and positivity

I will maintain this blog to make sure that it contains a low amount of code but to the musings of my fellow programmers or people who are interested in this project, I will add a few pieces of code. If you are interested in using the project as a starter code for your own, you can clone it from my GitHub, if you find anything interesting send me your findings, I will be interested in checking it out.

Analysis

Alright, so now that we have preliminary data, we will try to find and answer some questions and see what story our data will tell us.

When you are searching for an Airbnb for your vacation you would first look at the price and check what you would get for the price. This is a correlation. Let’s create a correlation between price and different features of the list.

• How do different features affect the price of a listing?

The stronger the correlation the darker the cell of the image above. Either negative or positive. Looking at the image above not surprisingly the number of people it can accommodate, beds, and bedrooms affect the price quite significantly. But surprising that the number of reviews, minimum nights, and a number of listings of that host do not affect the price in a significant way.

So now keeping that in mind, what are the best or the worst listings of Boston Airbnb?

• Best and worst listings?

So post our analysis the best listings on Boston Airbnb have a customer satisfaction rate of 0.5 and the worst satisfaction rate of 0.04 (don’t be deceived by the strength of the bar always look at the scale). We know some listings would be bad but 0.04? That is very bad. But how do we get customer satisfaction and how reliable is it?

Well, that question is fairly simple, we have “reviews” data for each of these listings and we know that we can do some simple text mining and make sure it is in a decent shape. We then use these reviews to perform sentiment analysis on them to get a positive value, negative value, and neutral value. So what you see above in the image is the positive value or pos sentiment of the reviews that the listings have.

We still have to question the reviewers, some people are just very negative no matter what, aren’t they? Let’s check our top reviewers and how positive their opinions are.

• How do our top reviewers usually feel about their stay?

We sort our data to get the top reviewers by the number of reviews they have individually given and perform sentiment analysis on their reviews and get the positivity rate and plot them. The top most plot consists of 30 top reviewers vs the number of individual reviews. The second and third plots are the corresponding sentiment positive rates based on two algorithms — Naive Bayes and pattern based sentiment.

• Alright then, so which months have the most bookings?

We do not exactly have this data so we improvise a bit. We take in the reviews written by reviewers and calculate the busiest bookings for each month. We should take this information with a pinch of salt.

Improvements

There are a couple of improvements that I would like to make eventually to this.

• I started working on this with the thought of generalizing the functions to suit datasets of different cities from AirBnB. This is something I will still work on, so check out my GitHub for new commits but for now, this would be just for Boston.
• I am working on clustering and geo plotting the different types of listings based on a few features.
• We can of course extract images of these listings and correlate listing images to bookings and how images help.
• More questions like host/super host preferences, seasonality, and trend of prices with local events.

Generalization

Of course, this existing code may create errors for other datasets even if we would like to generalize for every data we will have to check every column for null values or garbage data, which will take time. I may do it sometime later, but for now, use it as a starter code for your own analysis.

I will add more methods in the future to help this little project.

Conclusion

Thank you for reading my simple analysis of the Boston AirBnB dataset! If you like the blog do let me know and I will consider adding a part 2 for this blog with a few of the above features. Go over to my GitHub and check out the code for interactive plots used above and new feature commits.

Contents

1. Importing the dataset
2. Visualize and Clean the dataset
3. Dog Recognition
4. Create a CNN model from scratch
5. Create a CNN VGG16 model using Transfer Learning
6. Metrics
7. Classification using the above CNN models
8. Improvements
9. Conclusion

CNN or Convolutional Neural Network is a deep learning network of neurons that work exceptionally well on images. We will not dive into what CNN is or how it works but we will look at the application of CNN as a fun exercise for classifying dogs. Of course, as is the case for any machine learning or deep learning model we need huge data to solve the model parameter problem, that is find the best possible weights and bias for these models.

Importing the dataset

We will start with where all good machine learning projects begin, importing the dataset. We get the data and transform it into a numpy type array. We do the same with our targets, except the targets also need to be one hot encoded to convert them to usable input for our model.

def load_dataset(path):
    data = load_files(path)
    dog_files = np.array(data['filenames'])
    dog_targets = np_utils.to_categorical(np.array(data['target']), 133)
    return dog_files, dog_targets

# load train, test, and validation datasets
train_files, train_targets = load_dataset('../../../data/dog_images/train')
valid_files, valid_targets = load_dataset('../../../data/dog_images/valid')
test_files, test_targets = load_dataset('../../../data/dog_images/test')

There are 133 total dog categories.
There are 8351 total dog images.
There are 6680 training dog images.
There are 835 validation dog images.
There are 836 test dog images

Visualize and Clean the dataset

Let’s visualize one of our images

import matplotlib.pyplot as plt
import matplotlib.image as mpimg
plt.imshow(mpimg.imread(train_files[10]))

We will start by preprocessing the dog image to convert it to a tensor. We get a three-dimensional image and we will return a four-dimensional image by using the expand_dims method in numpy along the axis 0. We will also create a function to convert all our data into a list of tensors. The input would contain many images of various sizes as we see above with 400*390. To feed the images into our model we would like these tensors to be of the same shape. We pick 224*224 to standardize all our images.

def path_to_tensor(img_path):
    # loads RGB image as PIL.Image.Image type
    img = image.load_img(img_path, target_size=(224, 224))
    # convert PIL.Image.Image type to 3D tensor with shape (224, 224, 3)
    x = image.img_to_array(img)
    # convert 3D tensor to 4D tensor with shape (1, 224, 224, 3) and return 4D tensor
    return np.expand_dims(x, axis=0)

def paths_to_tensor(img_paths):
    list_of_tensors = [path_to_tensor(img_path) for img_path in tqdm(img_paths)]
    return np.vstack(list_of_tensors)

Let’s check for imbalance and plot

As we can see while there is an imbalance in different dog breed classes, we can still proceed with our preliminary models and see how the CNN models behave with this data.

unique, counts = np.unique(np.argmax(train_targets,axis = 1), return_counts=True)
print(dict(zip(unique, counts)))

{0: 64, 1: 58, 2: 52, 3: 63, 4: 77, 5: 64, 6: 50, 7: 66, 8: 34, 9: 50, 10: 66, 11: 66, 12: 46, 13: 69, 14: 73, 15: 59, 16: 62, 17: 50, 18: 48, 19: 62, 20: 64, 21: 47, 22: 65, 23: 62, 24: 37, 25: 41, 26: 64, 27: 35, 28: 74, 29: 52, 30: 56, 31: 65, 32: 45, 33: 64, 34: 53, 35: 65, 36: 50, 37: 57, 38: 69, 39: 53, 40: 69, 41: 63, 42: 50, 43: 64, 44: 53, 45: 67, 46: 54, 47: 54, 48: 50, 49: 50, 50: 62, 51: 49, 52: 47, 53: 57, 54: 50, 55: 65, 56: 71, 57: 50, 58: 47, 59: 60, 60: 61, 61: 53, 62: 53, 63: 39, 64: 42, 65: 33, 66: 34, 67: 63, 68: 51, 69: 47, 70: 62, 71: 48, 72: 42, 73: 41, 74: 44, 75: 64, 76: 43, 77: 40, 78: 59, 79: 46, 80: 56, 81: 61, 82: 46, 83: 50, 84: 37, 85: 53, 86: 66, 87: 51, 88: 53, 89: 58, 90: 57, 91: 44, 92: 35, 93: 44, 94: 49, 95: 43, 96: 50, 97: 46, 98: 42, 99: 34, 100: 48, 101: 29, 102: 58, 103: 42, 104: 31, 105: 50, 106: 46, 107: 26, 108: 45, 109: 33, 110: 44, 111: 54, 112: 39, 113: 35, 114: 63, 115: 30, 116: 48, 117: 53, 118: 31, 119: 39, 120: 28, 121: 32, 122: 44, 123: 50, 124: 34, 125: 30, 126: 41, 127: 30, 128: 48, 129: 44, 130: 30, 131: 26, 132: 30}

Dog Recognition

Since we are interested in classifying dogs, we might as well detect if there are dogs at all in the picture before we classify which breed the dog is. To do this we can either build a model from scratch or we can use a model like ResNet50 to classify True or False which indicates the presence of a dog. We will need tensors to process our data into a tensor, so we will use the above-defined path_to_tensor function to do this.

def ResNet50_predict_labels(img_path):
    # returns prediction vector for image located at img_path
    img = preprocess_input(path_to_tensor(img_path))
    return np.argmax(ResNet50_model.predict(img))

We can then use the above function to predict the presence of a dog. If the prediction label is between 151 and 268(both inclusive), the classes can be found in imagenet class list, we know that the image has a dog.

def dog_detector(img_path):
    prediction = ResNet50_predict_labels(img_path)
    return ((prediction <= 268) & (prediction >= 151))

Create a CNN model from scratch

We will create a simple CNN with 3 layers of Conv2D along with max-pooling and the last layer being a dense layer for final classification. Let’s first define our network.

model = Sequential()
model.add(Conv2D(filters=16, input_shape=(224, 224, 3), kernel_size=2))
model.add(MaxPooling2D(2))
model.add(Conv2D(32, 2, activation="relu"))
model.add(MaxPooling2D(2))
model.add(GlobalAveragePooling2D())
model.add(Dense(133, activation="softmax"))

model.summary()

We will then compile our model, set the checkpoint, and train the model on the above tensors. The number of epochs can really depend on you and on experimentation here epochs variable is set to 30

model.compile(optimizer=’rmsprop’, loss=’categorical_crossentropy’, metrics=[‘accuracy’])

checkpointer = ModelCheckpoint(filepath='saved_models/weights.best.from_scratch.hdf5', 
                               verbose=1, save_best_only=True)

model.fit(train_tensors, train_targets, 
          validation_data=(valid_tensors, valid_targets),
          epochs=epochs, batch_size=20, callbacks=[checkpointer], verbose=1)

Let’s load the model from the checkpoint and test it on the test data.

To test the accuracy we will get the predictions and divide the number of positives by the length of the data points in our test_data. Multiple with 100 and we have a percentage of the test accuracy on the model. Since this is a model from scratch we would not get great accuracy and is around 5 percent.

model.load_weights('saved_models/weights.best.from_scratch.hdf5')

dog_breed_predictions = [np.argmax(model.predict(np.expand_dims(tensor, axis=0))) for tensor in test_tensors]

test_accuracy = 100*np.sum(np.array(dog_breed_predictions)==np.argmax(test_targets, axis=1))/len(dog_breed_predictions)
print('Test accuracy: %.4f%%' % test_accuracy)

We will follow similar steps for other models which are pre-trained like VGG16, ResNet50, and InceptionV3.

Create a CNN VGG16 model using Transfer Learning

To use and train the VGG16 model we will have to first import the model weights and bias. we will also import the bottleneck features data for the training, validating, and testing. The rest of the process is pretty similar to what we have seen above and is going to follow a similar pattern. This is called transfer learning, we use a pre-trained VGG16 model (we do this using the features extracted from the training set using the VGG16 model), we call these bottleneck features, and we then define a few layers that we build to suit our problem.

bottleneck_features = np.load('bottleneck_features/DogVGG16Data.npz')
train_VGG16 = bottleneck_features['train']
valid_VGG16 = bottleneck_features['valid']
test_VGG16 = bottleneck_features['test']

VGG16_model = Sequential()
VGG16_model.add(Conv2D(filters=16, kernel_size=2, input_shape=train_VGG16.shape[1:]))
VGG16_model.add(Dropout(0.4))
VGG16_model.add(GlobalAveragePooling2D(input_shape=train_VGG16.shape[1:]))
VGG16_model.add(Dense(133, activation='softmax'))

VGG16_model.summary()

The rest of the steps are exactly as we define them above, we compile the model, save checkpoints, load weights, and calculate accuracy.

ResNet50 and InceptionV3 follow the same exact method as the VGG16 model, but we will need different bottleneck features which were extracted from their respective models. Click on the GitHub link below to see the code and usage of these models along with their architectures.

Metrics

As we saw above the imbalance in classes is not too high for us to get unusually high accuracy score. Which makes it okay(not perfect) to just use accuracy for this particular problem. We could use other metrics like F1 score, Recall, Precision in addition to accuracy to get a better understanding of the performance of the model.

The accuracies of all the models are — VGG16 at 71%, ResNet50 at 80.26%, and InceptionV3 at 80.02%.

Classification using the above CNN models

Finally, once we are done training our models, we will load the model weights into the environment. I am using a small function called dog_breeds to take in an image_path as an input.

def dog_breeds(image_path):
    from keras.models import load_model
    VGG19_model = load_model('saved_models/weights.best.VGG16.hdf5')
    Resnet50_model = load_model('saved_models/weights.best.Resnet50.hdf5')
    InceptionV3_model = load_model('saved_models/weights.best.InceptionV3.hdf5')
    image_tensor = path_to_tensor(image_path) 

We can use these model objects to predict on the image_path of a dog. The below snippet is written in the dog_breeds function. Compile the results into a list and return them for our convenience.

results = [np.argmax(VGG19_model.predict(extract_VGG19(image_tensor))),
     np.argmax(Resnet50_model.predict(extract_Resnet50(image_tensor))),
     np.argmax(InceptionV3_model.predict(extract_InceptionV3(image_tensor))),
     ]
    
    return [dog_names[i] for i in results]

Improvements

A fun exercise within this project is that we can detect if an image has a human face and pass that image to the dog_breeds function to find which dog breed the human face is classified into. Similar to the dog_detector function, but in this case we use the Haar Cascades classifier with the human images to detect a human face.

def human_or_dog(image_path=None):
    if image_path==None:
        return("Error, human_or_dog: File path cannot be None")
    else:
        if dog_detector(image_path):
            return(dog_breeds(image_path))
        elif face_detector(image_path):
            return(dog_breeds(image_path))
        else:
            return("Error, human_or_dog: Image does not have a face")

An example output is, that the below image is classified as a Pekingese.

• There are more improvements that can be done in this project. Using image augmentation is one of them and would probably give better results than the models above.
• Hyperparameter tuning to tune the number of epochs, batches, and layers will also result in better accuracy than the above models.

Conclusion

The notebook for this blog is available on my GitHub. Feel free to reach out for any comments or suggestions.

In this article, I want to highlight in the easiest way possible how to achieve Style Transfer a concept that uses AI in the art to achieve some astonishingly beautiful results.

Contents

1. What is Style Transfer?
2. What will we use?
3. Where do we start?
4. Deep Learning model
5. Loss Functions
6. Calculating gradients
7. Optimization
8. Reverse the preprocessing on the resulting image
9. Sources and credits

What is Style Transfer?

Style transfer is a concept which basically adopts the visual style of one image on another image. Let’s take a classic example of a picture of Katsushika Hokusai’s great wave image as a reference style image and a dog image as the content image combining together should result in the dog image shown with the style of the great wave image.

Now without further ado, I will show you how to create this program and I am sure you will find it not difficult considering the enigma displayed by a lot of people toward AI or deep learning in particular.

On Artificial Intelligence and Surveillance Capitalism | Data Driven Investor

What will we use?

For this application, we will use Keras for backend and deep learning model, PIL for image processing, scipy for an optimization algorithm and time for seeing how long our training process takes for each iteration.

But I don’t know Keras. What do I do?

Even if you are not familiar with Keras you should be able to understand this because of the ease of the syntax and I will try my best to explain the code in as simple words as possible.

Where do we start?

We will use a Jupyter notebook so let’s import some basic necessities and also the images that we want to work with into the environment.

We convert these images to numpy arrays. We will add an extra dimension to these numpy arrays so that we create a tensor with the content, style and the combination data for using it with the deep learning model.

Because we will be using the pre-trained VGG16 model we will need to preprocess our input images a little bit. We will subtract every RGB channel with the pre-calculated mean values with the image net dataset.

Now reverse RGB to BGR to match the architecture VGG16 which was written in Caffe.

We now convert each of our variables into tensors. We also create our combination variable to hold our combination image when created. As said before we concatenate all these tensors to create one input tensor to the deep learning model.

Deep Learning model

Alright so after all our input preparation we finally come to the part where we import our deep learning model VGG16 and play with some layers.

The VGG16 model was created for a classification problem but we are not doing that so we don't need some layers called the fully connected layers which are responsible for this classification. We can exclude them by using the include_top parameter. Creating a dictionary as you see in the last line of the code snippet to access each of these layers by their layer name.

The plan with the VGG16 model is to make the model “understand” the features of images. We use this to our advantage to find the loss between the content image and the combination image and also between style image and combination image and calculate gradients to minimize the loss.

So let’s go ahead and code our loss functions.

Loss Functions

Considering what we have seen before about VGG16 models being able to encode the perceptual features of the images. We will use some of its layers to achieve this. You can read about the model layers in original paper from Gatys et al., But we will implement the layers from Johnson et al. (2016)

Content Loss

Loss between content image and the combination image is just the MSE between the two variables.

Style Loss

Style loss is not exactly the way we calculate content loss. We want to capture the style of both these images. The way we do this is by calculating the gram matrix which is the dot product of the style image variable with its transpose.

This transformation brings the focus on the style than the content of the image.

Total Variation Loss

Now we calculate the total variation loss to increase or decrease the noise in the loss term which we will see how it does.

But before we do that we would want to give these three losses some weight so we can vary the influence of each of these losses on our total loss term.

Let’s create a loss variable and add all these losses up.

Calculating gradients

We have everything we need in place. All we now need to do is call an optimizer to iterate and optimize the combination image. Let’s create our gradients for the loss with respect to the combination variable. In this particular code I am using TF1 because I like the gradients function but in TF2 gradients eager execution is not supported(not sure if it is a bug or a feature) so I encourage you to try out the GradientTape function.

For a better understanding of backend.function take a look at this stack overflow post.

We will evaluate our loss and gradients in the function to make sure we have the right values.

To make our lives easier with the optimizer we will create a class and return the loss value and grad value which the optimizer will try to modify to bring out the most optimum result.

Optimization

So let’s code our optimization part. We will use the fmin_l_bfgs_b algorithm from scipy. We create a variable with random data and store the created image in this variable.

Reverse the preprocessing on the resulting image

We now reverse what we have done before. Convert BGR to RGB values and we add the mean values to the image. We clip our values to be between 0 and 255 pixel values.

And now we see our resulting image.

Sources and credits

1. https://harishnarayanan.org/writing/artistic-style-transfer/ one of the best blogs on style transfer. Thanks to Harish Narayan for such a wonderful explanation. It helped me with practicing style transfer.
2. https://keras.io/examples/neural_style_transfer/ Keras documentation and Keras code for NST is just amazing.

Github repo for this notebook or take a look at the notebook from this page.

I would like to know your thoughts and views on my blog. Constructive criticism is much appreciated. If you like this blog and you can buy me a coffee by clicking the button below.

Neural Style Transfer (AI in art) with Keras and TF2 was originally published in DataDrivenInvestor on Medium, where people are continuing the conversation by highlighting and responding to this story.

What is data science?

Before we answer why, we have to understand what data science actually is. According to Wikipedia “Data science is a multi-disciplinary field that uses scientific methods, processes, algorithms, and systems to extract knowledge and insights from data in various forms, both structured and unstructured, similar to data mining.”

Simply put data science is a field where data in its raw form is processed into information.

What does domain knowledge mean?

The term “Domain Knowledge” has been in play even before data science became popular. In software engineering, it means the knowledge about the environment in which the target (i.e. software agent) operates.

We can use the same definition in data science to say — “Domain knowledge is the knowledge about the environment in which the data is processed to reveal secrets of the data”. In other words, the knowledge of the field that the data belongs to is known as Domain Knowledge.

How does domain knowledge influence data science?

You may have studied data science and machine learning and used some machine learning algorithms like regression, classification to predict on some test data. But the true power of an algorithm and data can be harnessed only when we have some form of domain knowledge. Needless to say, the accuracy of the model also increases with the use of such knowledge of data.

For example, the knowledge of the automobile industry when working with the relevant data can be used like — Let’s say we have two features Horsepower and RPM from which we can create an additional feature like Torque from the formula

TORQUE = HP x 5252 ÷ RPM

This could potentially influence the output when we train a machine learning model and result in higher accuracy.

Where is domain knowledge useful?

You may have understood from the above example that domain knowledge is best useful in feature engineering. Feature engineering is creating features using the domain knowledge to optimize the machine learning algorithms.

Let’s see an example in the economics related data to support what we have seen so far. The combination of economics and mathematical concepts is called Econometrics and machine learning particularly regression is being used widely these days to create insights using the raw data. We see two models, one without feature engineering and one with feature engineering using domain knowledge of economics. To keep this blog simple and concise we will only fit the models and compare them.

For example, let’s take the Catalonia GDP data which you can download here. Let’s see the head of the dataset

We will then apply linear regression before and after applying domain knowledge.

Without application of domain knowledge

Output

0.9905101700533853
3325.856952652541

With the application of domain knowledge

We apply domain knowledge in creating features like trade openness by combining two features total exports and total imports and domestic demand per GDP without construction by subtracting construction sector from domestic demand and dividing the result with GDP as shown below.

And finally, fit the model and check its scores.

Output

0.9961357892269006
2122.2901303188205

Conclusion

In conclusion, we see that using feature engineering by applying domain knowledge gives better accuracy score and lesser RMSE than the model without. You can get the entire code here.

References

1. Catalonia GDP kaggle kernel

Contact

You can give me your feedback in the comments below or email me at saianand0427@gmail.com. I am on Kaggle and GitHub as well!

I recently made a network graph using plotly on AT&T data and it turned out to be beautiful so I thought why not share it with everyone.

This will be a detailed tutorial on how to make a network graph. In addition to plotly, we will also be using networkx to construct the graph. You can download the dataset from kaggle.

Network graph helps in visualizing connections between different nodes. This, in turn, would help us with network analysis. So without further ado lets discuss our data and see how we can plot it.

When we see the data there are two columns with source IP and destination IP which represent our edges. And each of these unique IP addresses is a node.

We use networkx to build our graph and include our nodes and edges in the graph and then pass on the nodes and edges to plotly to plot.

A = list(network_df["source_ip"].unique())
B = list(network_df["destination_ip"].unique())
node_list = list(set(A+B))
G = nx.Graph()
for i in node_list:
    G.add_node(i)

In the above for loop, we add all the nodes into the graph using the add_node method. Now we add edges to the graph.

for i,j in network_df.iterrows():
    G.add_edges_from([(j["source_ip"],j["destination_ip"])])

We now assign some position to each node for plotting. We can do so using a very convenient method called spring_layout in networkx.

pos = nx.spring_layout(G, k=0.5, iterations=50)
for n, p in pos.items():
    G.node[n]['pos'] = p

Now that our graph is set, we are ready to plot using plotly.

edge_trace = go.Scatter(
    x=[],
    y=[],
    line=dict(width=0.5,color='#888'),
    hoverinfo='none',
    mode='lines')

for edge in G.edges():
    x0, y0 = G.node[edge[0]]['pos']
    x1, y1 = G.node[edge[1]]['pos']
    edge_trace['x'] += tuple([x0, x1, None])
    edge_trace['y'] += tuple([y0, y1, None])

node_trace = go.Scatter(
    x=[],
    y=[],
    text=[],
    mode='markers',
    hoverinfo='text',
    marker=dict(
        showscale=True,
        colorscale='RdBu',
        reversescale=True,
        color=[],
        size=15,
        colorbar=dict(
            thickness=10,
            title='Node Connections',
            xanchor='left',
            titleside='right'
        ),
        line=dict(width=0)))

for node in G.nodes():
    x, y = G.node[node]['pos']
    node_trace['x'] += tuple([x])
    node_trace['y'] += tuple([y])

In the code above we create go objects of plotly and assign the data to the objects. Now we assign annotations to the nodes and edges.

for node, adjacencies in enumerate(G.adjacency()):
    node_trace['marker']['color']+=tuple([len(adjacencies[1])])
    node_info = adjacencies[0] +' # of connections: '+str(len(adjacencies[1]))
    node_trace['text']+=tuple([node_info])

Now we plot go objects created above node_trace and edge_trace using Figure.

fig = go.Figure(data=[edge_trace, node_trace],
             layout=go.Layout(
                title='<br>AT&T network connections',
                titlefont=dict(size=16),
                showlegend=False,
                hovermode='closest',
                margin=dict(b=20,l=5,r=5,t=40),
                annotations=[ dict(
                    text="No. of connections",
                    showarrow=False,
                    xref="paper", yref="paper") ],
                xaxis=dict(showgrid=False, zeroline=False, showticklabels=False),
                yaxis=dict(showgrid=False, zeroline=False, showticklabels=False)))
iplot(fig)

The output is an interactive network graph which could be saved as a png, HTML and many other forms. For the sake of using the interactive nature, I saved it as an HTML and uploaded to plot.ly. Check out the below link which is the output of the above code.

AT&T network connections | line chart made by Anand0427 | plotly

So that's it! We created a network graph. The code for it is available on my Kaggle and Github. Let me know what you think below. You can always hit me up on my mail ID saianand0427@gmail.com. Clap and share! See you in my next blog!

Welcome to Bokeh - Bokeh 1.2.0 documentation

Bokeh is available for Python and JavaScript. In this tutorial, I am going to cover Bokeh for Python. Go to this link on how to install Bokeh for python.

It should not really matter which IDE you use so feel free to use whatever IDE you like. Once you are done installing Bokeh in your computers try importing bokeh in your IDE.

import bokeh

In bokeh the points or makers on a graph are called glyphs. Read more about glyphs here.

Lets mark circle glyphs on a plot.

from bokeh.io import output_file,show,output_notebook

Bokeh.io is for the IO operations like export the file or display it in notebook etc. We will see how this works shortly. Below line is the actual code which is our figure.

from bokeh.plotting import figure

Lets go ahead and create a figure. For the sake of simplicity and to best show how bokeh works I chose the AirPassengers dataset. I did make some small additions to the dataset like sum and average of the number of passengers flying per month. I will link my Kaggle kernel at the end of this tutorial for some basic analysis on the data.

fig = figure(x_axis_label = "Year",y_axis_label = "Average and Max of Air Passengers per Year")

Now that we have our figure ready we plot our circle glyphs as follows.

fig.circle(df["Year"],df["Avg"])

Now our figure has circle glyph in it and we need to display it.

output_notebook()
show(fig)

Output_notebook is for specifying to display in our notebook. This will give us the following plot.

One of the special things about bokeh is existing plots can be drawn upon with other plots. Observe the following code and its output.

fig.line(df["Year"],df["Avg"])
show(fig)

In the above plot we see the line glyph on top of the circle glyphs. This makes it easier for the developers to plot multiple plots easily without writing too many lines of code or too many subplots. Another example of the same below.

fig.x(df["Year"],df["Max"],line_color="red",size=10)
fig.line(df["Year"],df["Max"],line_color="red")
show(fig)

In this plot along with out previous one we plot the ‘x’ glyph and the line glyph together.

Also to be noted is I changed the column from “avg” to “Max” which is the reason we see a different line. But the label is still the same because our figure method is common to all the glyphs.

Thats all for now. Try and figure out if you can plot the below figure. You can look for the patches glyph in the bokeh docs.

The dataset used for this tutorial is available here. My Kaggle kernel for reference and the code as solution to the patches glyph.

Do give me your feedback and suggestions in the comments below. If you would like to see more of Bokeh let me know too. If you need help please feel free to ask anytime.

To those of you who didn't get a chance to attend the roadshow here is a brief summary of the event at Hotel Conrad in Bengaluru.

First of all click here for the schedule and here for the awesome speakers list.

All of the speakers were from the TensorFlow team and spoke about some interesting topics which I will list out below.

1. TensorFlow with eager execution to train and package a saved model. Cool resource for it.

2. Autographing for writing complex models easily while also not giving up on the graph mode. More on this here.

3. Distribution strategies with TensorFlow distribute API for spreading across GPU or TPU.

4. Introduction and evolution to the Tensor Processing Unit and performance improvement compared to the GPU.

5. End to end platform using Tensor flow extended. Still being developed but check out the examples.

6. TensorFlow beyond Python so TensorFlow for Swift.

7. Building models using TensorFlow.js which is basically same as the TensorFlow in Python but on the browser.

8. TensorFlow Lite, lightweight TensorFlow for devices which have less power.

9. Tensor2Tensor a library of deep learning models and datasets.

10. Lastly and my favorite of all. GANS. This talk was about how to enable the machine to learn the patterns in music and to generate new music with some input. Also spoke about text generation and autoencoders. No particular link for this but check out DCGANs.

Overall it was a good day and looking forward to the next one. Let me know what you think. Also If you were there and I missed any let me know in the comments below.

Thanks for reading.