-
Abstract
+ {% endif %} {{ content }} diff --git a/_pages/about.html b/_pages/about.html index 896274cc..dcb05b30 100644 --- a/_pages/about.html +++ b/_pages/about.html @@ -3,26 +3,20 @@ title: About permalink: /about/ --- -Good day fellow readers and welcome to my tech blog !
+Welcome to Weiwei's Tech Blog!
-My name Alain Pham. -I am currently working as a Solution Architect specialized in Application Development. -My main task is to deliver complex proof of concepts across EMEA to enable the adoption of open source technologies. -
+-The purpose of my life? -Since a very young age I have always been passionate about ecosystems of technology. So my purpose on earth is -making it possible to advantage of a wide range of technologies and methods and to solve one complex problem after another.
-You can also follow me on social media:
-
+You can also follow me on social media:
+
-
+
-
-
+
Many organisations are now facing a challenge when it comes to - choosing and setting up the right messaging infrastructure. It must be - able to scale and handle massive parallel connections. This challenge - often emerges with IoT & Big Data projects where a massive - number of sensors are potentially connected to produce messages that - need to be processed and stored.
-This post explains how to address this challenge using the concept of network brokers in JBoss Active MQ.
- - -An important thing to bare in mind when speaking of scaling the messaging infrastructure is that one must consider 2 distinct capabilities --
-
- scaling up when there is heavy load -
- scaling down to save costs when compute power is not need -
At the time of writing the uses cases in this post have been tested with JBoss Active MQ 6.3.0
-Choosing the right topology
- -Choosing the right broker network topology is a crucial step. - -There are many examples of network topology that can be found in the official JBoss ActiveMQ documentation : - - - https://access.redhat.com/documentation/en/red-hat-jboss-a-mq/6.3/paged/using-networks-of-brokers/chapter-6-network-topologies - - -
- --To have a better understanding, we need to have the following concepts in mind. -Each broker can have : -
--
-
- 1 or many transport connector to accept client connections (consumers or producers) and network connections from other brokers (receiving messages from other brokers). In our sample here we only have 1 transport connector per broker instance -
- 1 or many network connectors to allow messages to be forwarded to other brokers -
In this article we will take the example of an IoT use case where the main focus is on :
--
-
- handing a large number of connected sensors -
- securing a central system that collects and processes messages -
In such scenario, we would like to set up a concentrator topology as shown below.
- - -
--This topology allows us to setup stable and controlled connections between the brokers of layer 1 towards layer 2. The central brokers do not have to deal with sensors connecting and disconnecting. - -The green arrows define a network connector. Note that these have a direction. It means that brokers from layer 1 can forward messages to all brokers from layer 2. - -We also allow messages to travel between brokers of layer 2. We will see later that this allows the network to better balance the workload and to facilitates scale ups/scale downs. -
-How to scale up ?
-Scaling up is pretty straight in this setup : --
-
- To handle more connections, brokers can be added in layer 1 -
- If the overall volume grows, extend the number of brokers in layer 2 to absorb the workload -
-Usually when scaling up, we want the clients already connected to be redistributed among the newly available brokers. This is especially important for the clients connected to the layer 2 as we might not want to statically bind them to one broker. - -To achieve this we can set the following options within the transport connectors of the central brokers. -
--
-
- rebalanceClusterClients : allows the broker to disconnect clients and force them to reconnect to a less busy broker -
- updateClusterClients : updates client connections when a broker joins the network. In that way the clients do not need to be aware of all the brokers available when connecting for the first time. The clients only need to be able to connect to at least one. -
- updateClusterClientsOnRemove : updates client connections when a broker leaves the network. -
- updateClusterFilter : a regular expression that allows to trigger client updates only on certain brokers that join the network. In our case for example we only want brokers from layer 2 to trigger reconnects. -
-Further options on the transport connector can be found here : - -https://access.redhat.com/documentation/en/red-hat-jboss-a-mq/6.3/paged/connection-reference/appendix-c-server-options -
-How to scale down?
- -Scaling down is a bit more challenging. This is because when shutting down an instance, there might be still messages remaining in the broker. - -The strategy we can apply to shut down a broker (named c01 in this example) properly in a network is the following : -
--
-
- Stop the transport connector through JMX/Jolokia of broker c01. This has the effect of disconnecting all consumers and producers from broker c01. It also disconnects all incoming network connectors. -
- The clients (producers & consumers) will fail over to an active broker instance -
- Unconsumed messages in broker c01 will be forwarded to another broker instance since the network connect of broker c01 towards its neighbor is still active -
- Once all messages in broker c01 are consumed, we can shut down the entire instance -
--There might be situations where this procedure is not enough. Messages may remain when no active consumers for those exist anywhere in the network. In this case, I recommend you to look at Josh Reagan's blog post: - -http://joshdreagan.github.io/2016/08/22/decommissioning_jboss_a-mq_brokers/. -It describes how to read the message store to be decommissioned and forward all remaining messages. -
-How is the load distributed ?
-Deactivate conduit subscriptions on queues for better load balancing
-When messages are sent to brokers in layer 1, by default these are distributed in a round robin manner to brokers in layer 2. - -For better load balancing on queues, conduit subscriptions can be deactivated on the brokers of layer 1. If consumers are not evenly distributed on layer 2, it allows brokers from layer 1 to be aware of that and apply the round robin algorithm on the different consumers rather than the brokers they are connected to. - -To have a deeper understanding you can refer to the following documentation : - -https://access.redhat.com/documentation/en/red-hat-jboss-a-mq/6.3/paged/using-networks-of-brokers/chapter-9-load-balancing -Message hopping on layer 2 to avoid slow consumers
- - -
-
-If you run the tests that are provided in the sample code (see below). You might notice that sometimes a message that reaches a broker in layer 2 is forwarded to its neighbor on layer 2 instead of being consumed by the local client.
-
-This is a very powerful feature of Active MQ that allows messages to be consumed as fast as possible within the network.
-
-This usually happens when the local consumer is slowing down. A good messaging system is one that tends to contain 0 messages (all messages are consumed as soon as they are produced). So what we can observe here is that the network of brokers is doing it's best to move the messages around so that they are consumed as fast as possible.
-
-On the other hand though, the consequences of this is that messages might get stuck at some point. There is a rule saying that a message cannot go back to a broker that has already been visited. So imagine if the fast consumer suddenly disconnects, our message here will get stuck in c01.
-How to avoid stuck messages ?
-To avoid messages getting stuck we need to add the replayWhenNoConsumers policy on the queues that are used within the network. -{% highlight xml %} -Other important configurations
-To ensure that the messages are prioritizing the shortest paths, these options need to be set on the network connectors : --
-
- decreaseNetworkConsumerPriority="true" -
- suppressDuplicateQueueSubscriptions="true" -
Sample code & configurations
-Nothing is better than a working example. A sample project can be found in the following GitHub repo : https://github.com/alainpham/amq-broker-network - -This project populates 3 AMQ instances configured as a netwok of brokers similar to what is described in the sections above : - - - -
-
-
--
-
- s01 is the instance where the producers will connect to (layer 1). We only need on of those to understand the different concepts. This instance has 2 network connectors, one towards c01 and another one towards c02 -
- c01 and c02 are the instances where the central consumers connect to in order to process messages (alyer 2). They each have a network connector towards each other -
mvn test-
-
-
- nominalSend1000Msg
-
-
-
- create the network of 3 brokers with 1 producer and 2 consumers -
- send 1000 messages -
- each consumer receives roughly half of the messages -
- some messages hop between c01 and c02 when consumers are a bit slow -
-
[ main] TestAMQNetwork INFO Received on collector01 500 -[ main] TestAMQNetwork INFO Received on collector02 500 -[ main] TestAMQNetwork INFO Total received 1000 -[ main] TestAMQNetwork INFO amqs01 enqueue 1000 -[ main] TestAMQNetwork INFO amqs01 dequeue 1000 -[ main] TestAMQNetwork INFO amqs01 dispatch 1000 -[ main] TestAMQNetwork INFO amqc01master enqueue 501 -[ main] TestAMQNetwork INFO amqc01master dequeue 501 -[ main] TestAMQNetwork INFO amqc01master dispatch 501 -[ main] TestAMQNetwork INFO amqc02master enqueue 501 -[ main] TestAMQNetwork INFO amqc02master dequeue 501 -[ main] TestAMQNetwork INFO amqc02master dispatch 501-
-
-
- scaleDown
-
-
-
- create the network of 3 brokers with 1 producer and 2 consumers -
- send 500 messages -
- stop transport connector on c01 -
- consumer on c01 reconnects to c02, remaining messages on c01 are drained -
- send another 500 messages -
- each consumer receives roughly half of the messages -
-
############### before scaleDown ######################### -[ main] TestAMQNetwork INFO ################################### -[ main] TestAMQNetwork INFO ################################### -[ main] TestAMQNetwork INFO Received on collector01 250 -[ main] TestAMQNetwork INFO Received on collector02 250 -[ main] TestAMQNetwork INFO Total received 500 -[ main] TestAMQNetwork INFO amqs01 enqueue 500 -[ main] TestAMQNetwork INFO amqs01 dequeue 500 -[ main] TestAMQNetwork INFO amqs01 dispatch 500 -[ main] TestAMQNetwork INFO amqc01master enqueue 250 -[ main] TestAMQNetwork INFO amqc01master dequeue 250 -[ main] TestAMQNetwork INFO amqc01master dispatch 250 -[ main] TestAMQNetwork INFO amqc02master enqueue 250 -[ main] TestAMQNetwork INFO amqc02master dequeue 250 -[ main] TestAMQNetwork INFO amqc02master dispatch 250-
############### scaleDown ######################### -[ main] TestAMQNetwork INFO ############################## -[ main] TestAMQNetwork INFO ############################## -[ main] TestAMQNetwork INFO Received on collector01 501 -[ main] TestAMQNetwork INFO Received on collector02 499 -[ main] TestAMQNetwork INFO Total received 1000 -[ main] TestAMQNetwork INFO amqs01 enqueue 1000 -[ main] TestAMQNetwork INFO amqs01 dequeue 1000 -[ main] TestAMQNetwork INFO amqs01 dispatch 1000 -[ main] TestAMQNetwork INFO amqc01master enqueue 250 -[ main] TestAMQNetwork INFO amqc01master dequeue 250 -[ main] TestAMQNetwork INFO amqc01master dispatch 250 -[ main] TestAMQNetwork INFO amqc02master enqueue 750 -[ main] TestAMQNetwork INFO amqc02master dequeue 750 -[ main] TestAMQNetwork INFO amqc02master dispatch 750 - --
-
-
- scaleUp
-
-
-
- c01 is not started at the beginning, only s01 and c02 are up. -
- send 500 msgs -
- start c01 -
- one of the consumer is forced to reconnected to c01 -
- send another 500 msgs -
- each consumer receives roughly half of the messages -
- c01 starts to broke messages once it's up -
-
############### before scaleUp ######################### -[ main] TestAMQNetwork INFO Received on collector01 250 -[ main] TestAMQNetwork INFO Received on collector02 250 -[ main] TestAMQNetwork INFO Total received 500 -[ main] TestAMQNetwork INFO amqs01 enqueue 500 -[ main] TestAMQNetwork INFO amqs01 dequeue 500 -[ main] TestAMQNetwork INFO amqs01 dispatch 500 -[ main] TestAMQNetwork INFO amqc01master enqueue 0 -[ main] TestAMQNetwork INFO amqc01master dequeue 0 -[ main] TestAMQNetwork INFO amqc01master dispatch 0 -[ main] TestAMQNetwork INFO amqc02master enqueue 500 -[ main] TestAMQNetwork INFO amqc02master dequeue 500 -[ main] TestAMQNetwork INFO amqc02master dispatch 500 - --
############### scaleUp ######################### -[ main] TestAMQNetwork INFO Received on collector01 500 -[ main] TestAMQNetwork INFO Received on collector02 500 -[ main] TestAMQNetwork INFO Total received 1000 -[ main] TestAMQNetwork INFO amqs01 enqueue 1000 -[ main] TestAMQNetwork INFO amqs01 dequeue 1000 -[ main] TestAMQNetwork INFO amqs01 dispatch 1000 -[ main] TestAMQNetwork INFO amqc01master enqueue 252 -[ main] TestAMQNetwork INFO amqc01master dequeue 252 -[ main] TestAMQNetwork INFO amqc01master dispatch 252 -[ main] TestAMQNetwork INFO amqc02master enqueue 752 -[ main] TestAMQNetwork INFO amqc02master dequeue 752 -[ main] TestAMQNetwork INFO amqc02master dispatch 752- -Thanks for reading ! diff --git a/_posts/2017-02-02-quickstart-rest-microservice-with-spring-boot.html b/_posts/2017-02-02-quickstart-rest-microservice-with-spring-boot.html deleted file mode 100644 index d482b5ce..00000000 --- a/_posts/2017-02-02-quickstart-rest-microservice-with-spring-boot.html +++ /dev/null @@ -1,227 +0,0 @@ ---- -layout: default -title: "Quickstart Json RESTful microservice with Spring-Boot" -date: 2017-02-03 09:00:00 +0200 -published: 2017-02-02 09:00:00 +0200 -comments: true -categories: development -tags: [spring-boot, quickstart, rest, json, microservice, cloudnative, development] -github: "https://github.com/alainpham/rest-service" ---- - -
In this post you will learn how to create a RESTful microservice - from scratch using Spring-Boot, Eclipse & Maven.
- - -Spring boot is one of the very popular frameworks to build - microservices. In fact it has been chosen to be the standard for JBoss - Fuse (Fuse Integration Services 2.0) deployments on Openshift. It can - also be used for general java application purposes. You would be - surprised how easy it is to create standalone packages that are capable - of running production grade services.
- -Source code for this can be found here : -https://github.com/alainpham/rest-service -
-These are the steps of this tutorial
- --
-
- Create a new simple Maven project in Eclipse -
- Add dependencies to Spring Boot -
- Launch your service -
Prerequisites
- -As a prerequisite for this tutorial you will need the following - elements:
- --
-
- Java JDK -
- Eclipse with maven plugin -
- Maven -
Create a simple maven project
- -First you must create a simple maven project. For that you can use the Eclipse Wizard
- -
-
-Chose simple project to have an empty pom.xml file
- -
-
-Fill in the project details
- -
-
-Edit the pom.xml file
- -Add these parent, dependencies, and plugin blocs to your pom
- -{% highlight xml %} -Create Application class
- -{% highlight java %} -package com.app; - -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.EnableAutoConfiguration; -import org.springframework.boot.autoconfigure.SpringBootApplication; -import org.springframework.context.annotation.ComponentScan; -import org.springframework.context.annotation.Configuration; - - - -@SpringBootApplication -public class Application { - - public static void main(String[] args) { - SpringApplication.run(Application.class, args); - } - -} -{% endhighlight %} - -Create a business model
-{% highlight java %} -package com.app.model; - -public class Person { - - private Long id; - private String name; - - - public Long getId() { - return id; - } - public void setId(Long id) { - this.id = id; - } - public String getName() { - return name; - } - public void setName(String name) { - this.name = name; - } - - -} -{% endhighlight %} - -Create a service
-{% highlight java %} -package com.app.service; - -import java.util.HashMap; -import java.util.Map; - -import org.springframework.web.bind.annotation.*; - -import com.app.model.Person; - -@RestController -@RequestMapping(value="/svc/person") -public class PersonService { - - MapRun your application
- -Open a command prompt in the root folder of your project and run :
-{% highlight shell %} -mvn spring-boot:run -{% endhighlight %} - -You can now also build your package and run the flat jar file
-{% highlight shell %} -mvn package -java -jar target/rest-service-0.0.1-SNAPSHOT.jar -{% endhighlight %} - -Now open your browser and consume your microservice by entering the following url
-{% highlight shell %} -http://localhost:8080/svc/person/1 -{% endhighlight %} - -
-
-The next post will be about how to deploy such microservices packaged in Docker containers on Openshift
- -Thanks for reading !
diff --git a/_posts/2017-02-13-ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform.html b/_posts/2017-02-13-ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform.html deleted file mode 100644 index c292b766..00000000 --- a/_posts/2017-02-13-ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform.html +++ /dev/null @@ -1,219 +0,0 @@ ---- -layout: default -title: "Ansible Automation for Highly Available JBoss Fuse/A-MQ Integration Platform" -date: 2017-02-13 09:00:00 +0200 -published: 2017-02-13 09:00:00 +0200 -comments: true -categories: automation -tags: [jboss-fuse, ansible, automation, integration, ha, scalability] -github: "https://github.com/alainpham/ansible-role-jboss-fuse-amq-ha" ---- - -Most integration/service platform projects start small but -need to ensure high availability requirements and be scalable to handle growing workloads. -This article shows how to use Ansible to automatically provision servers with JBoss Fuse and JBoss ActiveMQ instances -to get a highly available service and messaging platform. -
- - --Minimal Highly Available Architecture -
- --What does a new born reliable Integration Platform look like? -It needs at least 4 machines and a shared storage (NFSV4 or GFS2) as described in the following paragraphs. -
- - -
-
-
-- Two machines with JBoss Fuse -
-Here we will run 2 active instances of JBoss Fuse. -We will deploy our services on both machines to get high availablity and also load balancing. -- Two machines with JBoss A-MQ -
-Here we will run 2 couples of failover configured A-MQ instances. - We link them together in a network of brokers. - In the nominal case, A-MQ A instance should be active on one machine and A-MQ B instance should be active on the other machine. - Their failover instances are placed symmetrically on the other machine. -
-- This configuration has several advantages : -
--
-
- The messaging load is distributed on both machines thus giving a better resource utilization than just having a unique failover configuration -
- Shorter downtime when one machine fails can be achieved. - - The failover mechanism takes time to perform since it needs to detect that the lock on - the store has been released. Disconnected clients can reconnect immediately to the other active instance in the network and render service while - the failover instance resumes pending messages of the crashed instance. - -
-One might argue that dedicating machines to A-MQ is an overkill at the beginning. Why not just put them also on -the same machines as the JBoss Fuse engines? -Well that can be true if messaging is not that highly demanding in our use cases, but limitations will appear quickly when -trying to scale the messaging infrastructure once the platform grows. In fact we would like to be able to scale your messaging -infrastructure independently from the application services hosted on JBoss Fuse instances. In other words, if we do not allocate resources to -A-MQ from the beginning, our new born might be crippled and not be able to grow in the future. -
- --Create an Ansible Role for automation -
- --Ansible is a powerful tool to automate IT infrastructure provisioning, application deployment, configuration management and continuous delivery. -It is an agentless system meaning that we do not have to install anything on the target machines. -The only thing that is needed is a standard ssh access. -
- --To get the cluster up and running in little time, I have created an Ansible role that automates the following tasks : - -
- --
-
- - Upload packages to target machines - -
- - Setup default users for Fuse/A-MQ - -
- - Setup different ports (Hawtio, JMX, ssh, Openwire..) and folders for Fuse/A-MQ - -
- - If A-MQ network of brokers is activated, it will configure network connectors from one A-MQ instance to another - -
- - https://github.com/alainpham/ansible-role-jboss-fuse-amq-ha - -
--Below are the default variables to run the role. -
- -{% highlight yml %} ---- -######################################################## -#This lists all used variables with their sane defaults - -#################################### -#### Generic variables -#################################### -#path to the Fuse or A-MQ package -src_package_path: ./distrib/ - -#temporary folder on target host to unzip the package -temp_folder: /home/user/tmp/ - -#Fuse or A-MQ package to be installed with version -artifact: jboss-fuse-karaf-6.3.0.redhat-187.zip - -#folder on target host -target_package_folder: /home/user/fuse - -karaf_ssh_port: 8101 -karaf_http_port: 8181 -rmi_server_port: 44444 -rmi_registry_port: 1099 - -#setup default users -users: - - - password: admin - roles: - - admin - username: admin - - - password: guest - roles: - - viewer - username: guest - -#################################### -#### A-MQ variables -#################################### - -#flag to choose whether to setup a network of A-MQ brokers -setup_amq_network: no - -#########The following variables shoud be redefined in host_vars -amq_broker_name: amq -amq_openwire_port: 61616 -amq_data_store_folder: ${karaf.data}/amq -#this is an example of the list of target brokers -networked_hosts: - - connect_to_host: amq-b - master: amq-b-master - slave: amq-b-slave - -{% endhighlight %} --In order to run it, we must create a playbook and redefine those variables for Fuse machines and each A-MQ host. -You can find an example play book in the test folder. -
- --├── ansible.cfg -├── distrib -> folder to put packages in -│ ├── jboss-a-mq-6.3.0.redhat-187.zip -│ ├── jboss-fuse-karaf-6.3.0.redhat-187.zip -│ └── readme.txt -├── group_vars -│ ├── all.yml -> all the default variables -│ ├── amq.yml -> all defaults related to A-MQ machines -│ └── fuse.yml -> all defailts related to Fuse machines -├── host_vars -> specific configuration to hosts. -│ ├── amq-a-master.yml -│ ├── amq-a-slave.yml -│ ├── amq-b-master.yml -│ └── amq-b-slave.yml -├── inventory -└── playbook.yml -- - - -
-How to scale up the infrastructure? -
--So now the question is how do we grow this new born infrastructure? -For JBoss Fuse instances, it is very straightforward, just add machines to the inventory and Ansible will take care of setting up this machine. -
-
-For Active MQ, we need to think first about the topology of our network. Depending on the use cases this might differ.
-I have written a previous post detailing the concentrator topology for IOT purposes. It also explains how to scale down a network of brokers properly.
-
-
-Scalable Network of Active MQ brokers for handing massive connections
-
-
-If you need your integration platform to grow larger and be highly elastic, consider using PaaS solutions such as -Openshift. I would say that as a general rule of thumb if you exceed 5 machines in your integration cluster, -it can already be interesting to consider the Openshift iPaaS approach. -
- - --What next? -
-You can learn the basics about Ansible go to https://www.ansible.com/get-started.
-
-At some point you might want to scale down your integration platform, you can go ahead and make an Ansible role to automate down scaling.
-Furthermore you could write Ansible scripts to automate application deployments to Fuse containers.
-
-Service and api platforms are mostly real time oriented and handle small amounts of data that can be easily processed in memory. -But for many legacy purposes and for content management systems, being able -to handle large sets of data is a very common requirement. -This article shows how to easily send and receive large data files through HTTP with JBoss Fuse using streams. -The main objective of being able to use streams is to avoid running into out of memory issues. -
- - --JBoss Fuse and its streaming capabilities -
- -As a beginner at JBoss Fuse (particularly camel), you might have faced a few issues when the content of - the message body was a stream. If the body needs to be read more than once, it will result in - an exception at the second read attempt. - This is obviously because by default a stream can only be read once. - It can be quite confusing at the beginning but this gives such great flexibility as to how to process the content. - The customizations for handling stream is powerful and allows advanced tuning. - As an example, this is what could be done with a stream : -
--
-
- Load all into memory : this is the most basic thing to do if you want to be able to read the content twice. - But it is not possible when the content size is too large. - -
- - Load the content bloc by bloc to apply processing : this is useful if processing needs to be applied to the content. - Note that the content could be of any format here (xml, csv, binary..) - -
- Applying routing condition based on the size of the stream : the content could be loaded partially into memory - to be routed through a messaging channel (i.e A-MQ). - But if the size of content reaches a certain limit it could be written as a file instead - to be transfered through a proper file transfer protocol. This would prevent - large content from breaking the message broker. - -
- -
Streaming data with camel-jetty, camel-http4 and file components
-Components such as the camel-jetty, the camel-http4 and the file components that we will use here, do not load the entire payload into memory for processing
-unless we force it explicitly.
-Instead they will set the message body to a stream object (a pointer) that can be passed through the camel route and
-be read only needed.
-You can find an implementation example using these components here :
-
-https://github.com/alainpham/large-http-streams
-
-Below is the workflow implemented in the code example. The content of the input file is never loaded entirely into memory. -
- -
-
-{% highlight xml %}
-
--We are streaming through the initial file from end to end. In between the streams is a wire-protocol which is HTTP. -In order to verify that we are really streaming and not loading everything -into memory, we can proceed as follows : -
--
-
- Set the heap size of the JVM low at about 500MB and send a 700MB file through -
- Watch that memory consumption actually stays stable -
-Otherwise, it will fail with an out of memory exception if at some point we try to convert the body to a String (i.e after the consumer camel-jetty), -Note also that there are some implicit type conversions such as from a File type to an output stream for the camel-http4 component. -
-The nice thing with JBoss Fuse is that although the task to handle streams is quite advanced, -the code stays very simple and maintainable compared to custom low level Java Code. -This is thanks to the reusability of the frameworks components and the smart implicit type conversions built into camel. - -We could now easily tweak this example to add complex patterns such as stream-parsing XML data (i.e with XTokenizer) and -route chunks towards other processing units. These units may run in multiple threads or engines for parallel processing and become therefore naturally scalable. -
- -
diff --git a/_posts/2017-02-16-faster-fuse-integration-service-s2i-openshift-deployments.html b/_posts/2017-02-16-faster-fuse-integration-service-s2i-openshift-deployments.html
deleted file mode 100644
index 368759b9..00000000
--- a/_posts/2017-02-16-faster-fuse-integration-service-s2i-openshift-deployments.html
+++ /dev/null
@@ -1,480 +0,0 @@
----
-layout: default
-title: "Faster Fuse Integration Service 2.0 S2I Openshift Deployments"
-date: 2017-02-22 01:00:00 +0200
-published: 2017-02-16 01:00:00 +0200
-comments: true
-categories: development
-tags: [jboss-fuse, fis, openshift, build-time, paas, docker]
-github: "https://github.com/alainpham/fis20-simple-svc"
----
-
--Openshift offers a whole pipeline to create container images directly from source code. -It is usable for general purpose Java applications as well as for Fuse Integration Service projects. -This post will present how to accelerate the build time of those images by setting up a local Nexus Repository and configuring application templates -to use this repository for the build process. The idea is not having to download from the Internet all the FIS maven dependencies when building a new project. -At the time of writing FIS 2.0 is available as tech preview. -
- - -(update 22/02/2017) February 21, 2017 : FIS 2.0 GA has been released.
-The method shown here doesn't use the MVN_MIRROR_URL parameter which was enabled in FIS 2.0. -This will work on FIS 2.0 and also FIS 1.0. -
- -Prerequisites
--
-
- Openshift installed -
- JDK 1.8+ installed -
- Maven 3.3+ installed -
-Setting up a Nexus repository -
-The reason why build times can be long with the source S2I pipeline is because each new application build requires to spin up a -S2I container to launch the maven goals in order to package your application. -Building the same application template after the first time is faster because the downloaded artifacts are reused. -But every time we create a new application, it will go through the same initial downloading process again. -That is why instead of having the S2I container download everything from Internet repositories each time, -we can setup a Nexus server that will proxy these repositories and store already downloaded artifacts locally. -Here we will show how to setup a local standalone Nexus2 repository -
- -
-
-Download Nexus 2 OSS and run it
--Download the Nexus Repository Manager OSS 2.x package on : -https://www.sonatype.com/download-oss-sonatype -
--Unzip it and run it : -
- --./nexus console -- -
Setup proxy for Red Hat GA and Early Access repositories
-Connect to the user interface with the user and password admin/admin123
- --http://localhost:8081/nexus --
- 
-
-Click on Repositories and add 3 new proxy repositories -
--
-
- - The Red Hat GA repository (jboss-ga-repository) : https://maven.repository.redhat.com/ga/ - -
- - The Red Hat Early Access repository (jboss-ea-repository) : https://maven.repository.redhat.com/earlyaccess/all - -
- - The Fuse source repo eventually for older versions of Fuse (fuse-source) : https://repo.fusesource.com/nexus/content/groups/public/ - -
- 
-
Setup public group for all proxy repositories
-Select the public group, add all the created proxy repositories and maven central.
-
- 
-
-That's it, you have successfully setup your local Maven repository that will keep a local copy of all downloaded artifacts. -
- --Create a Fuse project with the Java Spring Boot archetype for Openshift -
- --Now in order to test a deployment let's create a Fuse project. Before we begin make sure that you have these prerequisites -on your developer machine : -
--
-
- JDK 1.7+ installed -
- Maven installed -
Configure maven settings
- --Make sure to configure your maven settings.xml file and add the Red Hat GA and Early access repositories to it. -
- -{% highlight xml %} -Create a FIS Spring Boot project
- --mvn archetype:generate \ - -DarchetypeCatalog=https://maven.repository.redhat.com/ga/io/fabric8/archetypes/archetypes-catalog/2.2.195.redhat-000004/archetypes-catalog-2.2.195.redhat-000004-archetype-catalog.xml \ - -DarchetypeGroupId=org.jboss.fuse.fis.archetypes \ - -DarchetypeArtifactId=spring-boot-camel-xml-archetype \ - -DarchetypeVersion=2.2.195.redhat-000004 -- -
Add necessary dependencies to the pom.xml
- --Add these dependencies to the pom.xml file in order to create a JSON Rest Webservice. -
- -{% highlight xml %} -Create routes to expose a rest webservice
- -Open the camel-context.xml file to create the following RestConfigurations and routes.
-{% highlight xml %} -To avoid a tcp port conflict with the default Nexus port (8081), make sure to change the parameter "management.port" in -the resources/application.properties file. You can change it to 8086 for example.
- -Run the project locally to see if everything is fine. Go to the root folder of your project.
-mvn spring-boot:run- -
Call the service using a browser/
-http://localhost:8666/test- -
-Create a settings.xml file -
- -At the root of your project copy your current maven settings.xml file -(the one with all the repositories defined) and add a mirror bloc as follows. -This will be used later by Openshift S2I. It there to configure Maven to get artifacts from your local Nexus repository.
- -{% highlight xml %} -Deploy on Openshift using S2i
- -Connect to your Openshift instance using the "oc" command
-oc login -u developer- -
Import the FIS 2.0 image streams
--wget raw.githubusercontent.com/jboss-fuse/application-templates/application-templates-2.0.redhat-000026/fis-image-streams.json -oc create -f fis-image-streams.json -- -
Create an application template YAML file
-This is a very crucial step. Here we will tweak the application template json/yml file so that -the S2I takes into account the Nexus repository for the Maven build.
- -Get the template json file for FIS Spring-Boot applications. Optionally you can convert the json to a yml file which I find easier to read.
--wget https://raw.githubusercontent.com/jboss-fuse/application-templates/application-templates-2.0.redhat-000026/quickstarts/springboot-camel-template.json -- -
The important parameters to change in the template are the following. -
-{% highlight yml %} -- name: APP_NAME - displayName: Application Name - required: true - value: springboot-rest - description: The name assigned to the application. -- name: GIT_REPO - displayName: Git Repository URL - required: true - value: https://github.com/alainpham/fis20-simple-svc.git - description: The URL of the repository with your application source code. -- name: GIT_REF - displayName: Git Reference - value: master - description: Set this to a branch name, tag or other ref of your repository if you - are not using the default branch. -- name: APP_VERSION - displayName: Application Version - value: 0.0.1 -- name: IMAGE_STREAM_NAMESPACE - displayName: Image Stream Namespace - value: '' - required: false -{% endhighlight %} - -The most important change is the following MAVEN_ARGS parameter. It is where we point to the Nexus repo we created and where we use -the settings.xml file at to root of the project. -Note that the IP address here 172.17.0.1 is the one that belongs to the host machine bound to the docker network interface. Adapt -it to your situation if necessary.
- -{% highlight yml %} -- name: MAVEN_ARGS - displayName: Maven Arguments - value: package -DMVN_MIRROR=http://172.17.0.1:8081/nexus/content/groups/public/ -gs settings.xml -DskipTests -Dfabric8.skip -e -B -{% endhighlight %} - -You should also add the following parameter to tweak the heap size of the JVM used for the maven build. In some rare cases you could run into an out of heap space exception. -It is always good to keep control over this parameter.
- -{% highlight yml %} -- name: MAVEN_OPTS - description: maven options such as heapspace - value: "-Xmx1024m" -{% endhighlight %} - -Add the environment parameter MAVEN_OPTS in the sourceStrategy bloc of the buildConfig so it is taken into account in the Maven command
- -{% highlight yml %} -sourceStrategy: - from: - kind: ImageStreamTag - namespace: "${IMAGE_STREAM_NAMESPACE}" - name: fis-java-openshift:${BUILDER_VERSION} - forcePull: true - incremental: true - env: - - name: BUILD_LOGLEVEL - value: '5' - - name: ARTIFACT_DIR - value: "${ARTIFACT_DIR}" - - name: MAVEN_ARGS - value: "${MAVEN_ARGS}" - - name: MAVEN_ARGS_APPEND - value: "${MAVEN_ARGS_APPEND}" - - name: MAVEN_OPTS - value: ${MAVEN_OPTS} -{% endhighlight %} - -We should also add the service and routes to expose the Rest service to the external world
-{% highlight yml %} -- apiVersion: v1 - kind: Service - metadata: - name: "${APP_NAME}" - creationTimestamp: - labels: - component: "${APP_NAME}" - group: quickstarts - project: "${APP_NAME}" - provider: s2i - version: "${APP_VERSION}" - spec: - ports: - - name: rest-svc-tcp - port: 8666 - protocol: TCP - targetPort: 8666 - selector: - component: "${APP_NAME}" - deploymentconfig: "${APP_NAME}" - group: quickstarts - project: "${APP_NAME}" - provider: s2i - version: "${APP_VERSION}" - sessionAffinity: None - type: ClusterIP - status: - loadBalancer: {} - -- apiVersion: v1 - kind: Route - metadata: - name: "${APP_NAME}" - creationTimestamp: - labels: - component: "${APP_NAME}" - group: quickstarts - project: "${APP_NAME}" - provider: s2i - version: "${APP_VERSION}" - spec: - to: - kind: Service - name: "${APP_NAME}" - port: - targetPort: rest-svc-tcp - status: {} -{% endhighlight %} - -This is the final file I have used :
--wget https://raw.githubusercontent.com/alainpham/fis20-simple-svc/v1.0/openshift/solution.yml -- -
The working project with all the resources can be found here :
-{{page.github }} - -Deploy and run the application
-You can use the command line. Alternatively you can use the Openshift UI to import the application template and deploy it.
--oc create -f solution.yml -oc create oc new-app springboot-rest -- -
-
-Test your service at the following url
--http://springboot-rest-myproject.[yourIP].xip.io/test -- -
Note that the build takes only about 2 minute the second time you run it instead of 20 minutes (in my case with a not so fast internet connection). This is thanks to the fact that the maven dependencies are -taken from the local Nexus instead of being downloaded from the Internet.
- - - - -Other resources : - --
-
- Video tutorial showing the source S2I method for FIS 2.0 : https://vimeo.com/194416659 - -
- Tutorial with binary S2I method (fabric8 plugin) : Getting Started with Fuse Integration Service 2.0 Tech preview -
- (update 22/02/2017)Official FIS 2.0 GA documentation : https://access.redhat.com/documentation/en-us/red_hat_jboss_middleware_for_openshift/3/html-single/red_hat_jboss_fuse_integration_services_2.0_for_openshift/ -
-Building Integration and Services platform with JBoss Fuse is great. -It is even better when you add a distributed in memory data base solution such as JBoss Data Grid to the mix. -This article will show how to make both products work together using the out of the box camel-jbossdatagrid component. -It will show you how to setup JBoss Data Grid in library mode within a JBoss Fuse application and take advantage of features such -as queries on indexed fields (uses Lucene search engine) and a persistent cache store. -
- -We will be building services to store and retrieve some business events. In summary the steps in this tutorial are the following
--
-
- Create a normal Fuse project in JBoss Developer Studio with Spring DSL -
- Add necessary dependencies to project pom.xml file -
- Configure the cache container in library mode through an Data Grid xml definition file (infinispan.xml) -
- Configure a cache called "event" with an index (Lucene) and a cache store (H2 database as an example) -
- Configure an embedded H2 database in the Camel context -
- Create an POJO as a model of an event -
- Annotate the POJO to specify which fields should be indexed for queries -
- Create basic generic camel routes for getting and putting events into the cache -
- Create a query service with dynamic parameters -
- Test the services -
Prerequisites
--
-
- JDK 1.8+ installed -
- Maven 3.3+ installed -
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
-
-https://maven.apache.org/download.cgi
Versions used
--
-
- JBoss Fuse 6.3.0 -
- JBoss Developer Studio Integration Stack 10.1.0 (optional) -
- JBoss Data Grid Camel Library 6.5.1 (downloaded through maven dependency) -
https://developers.redhat.com/products/fuse/download/
--Access modes to JBoss Data Grid -
-JBoss Data Grid can be accessed through 2 different modes : Library Mode and Remote Server Mode. -
-In Remote Server Mode, Data Grid is installed as a standalone distant server and Fuse gets access to the objects through the Hot Rod client. -
--In library mode (or embedded), the Fuse engine uses it's own JVM memory to contain objects that is part of the Data Grid. -In other words the Fuse engine acts as if it was a node that is part of the Data Grid cluster. -This mode gives access to more advanced features such as transactions and locking. It is also very simple to setup indexing for advanced querying. -In this article we will explore the possibilities of Library Mode. The Remote Server Mode will be discussed in an other article. -
- --Create a Fuse project -
--
-
- Start by creating a normal Fuse project with a Spring Camel Context in JBoss Developer Studio - (Alternatively it can also be created with a maven archetype via command line) -
- In JBoss Developer Studio go to files -> New -> Fuse Integration Project -
- Name the project techlab-fuse-jdg-library-mode -
- Select runtime version 2.17.0.redhat-630224 -
- Start an empty project with Spring DSL -
- Click finish -
- 
-
- 
-
-Edit infos and dependencies in pom.xml file -
--
-
- Change groupid, artifact id to identify the project -
- Add dependency to camel-jbossdatagrid component : this enables invokation of Data Grid endpoints in a camel route -
- Add dependency to infinispan-embedded-query : this enables indexing and query capabilities on the embedded cache -
- Add dependency to h2 : as an example of cache store we are using an h2 database -
- Add dependency to camel-netty4-http, camel-swagger-java, camel-jackson : will be used to expose rest services to access the objects stored in the cache -
Configure Data Grid Cache Container
--
-
- Create the file infinispan.xml in the folder src/main/resources/infinispan/ of the Fuse project -
- In the xml file, declare a cache named "event" -
- Configure a persistence cache store so that events can survive a reboot. As an example here we will use an H2 Database. - We will configure the database later in the Camel context -
- Activate indexing to enable advanced queries using the lucene engine -
- Configure eviction so that objects get unloaded from memory when not used. This is to avoid running out of memory. - Evicted entries stay in the cache store and can be reloaded when needed -
In the Camel context file reference the Data Grid configuration as follows
-{% highlight xml %} - - - -Add a reusable endpoint to the camel context
- -{% highlight xml %} -Configure H2 data base
-In the Camel context file add the following beans to create an embedded H2 database
-{% highlight xml %} - - - -Create a model classe for events and annotate for indexation
- -Create a class and annotate it so that it is indexed by Data Grid. Properties with the @Field annotation are indexed
- -{% highlight java %} -package techlab.model; - -import java.io.Serializable; -import java.util.Date; - -import org.hibernate.search.annotations.Analyze; -import org.hibernate.search.annotations.DateBridge; -import org.hibernate.search.annotations.Field; -import org.hibernate.search.annotations.Indexed; -import org.hibernate.search.annotations.Resolution; - -@Indexed -public class Event implements Serializable{ - - private static final long serialVersionUID = 1L; - - private String uid; - - @Field(analyze=Analyze.NO) - @DateBridge(resolution=Resolution.HOUR) - private Date timestmp; - - @Field(analyze=Analyze.NO) - private String name; - - private String content; - - public String getUid() { - return uid; - } - public void setUid(String uid) { - this.uid = uid; - } - public Date getTimestmp() { - return timestmp; - } - public void setTimestmp(Date timestmp) { - this.timestmp = timestmp; - } - public String getName() { - return name; - } - public void setName(String name) { - this.name = name; - } - - public String getContent() { - return content; - } - - public void setContent(String content) { - this.content = content; - } - -} -{% endhighlight %} - -Create rest services to get and put entries into the cache
- -Use the rest DSL to create routes and expose services to do the basic operations
- -{% highlight xml %} -Create a query service with dynamic parameters
- -Define a rest service that allows to pass any http query parameters
-
(i.e http://localhost:7123/query/event/techlab.model.Event?timestmp=1462208399999&name=ended)
Create a service to run queries against the cache
- -Create the processor that will be used by the camel route. Note that this class is pretty generic and is suitable to any data model. -
- -{% highlight java %} -package techlab.dg; - -import java.beans.BeanInfo; -import java.beans.Introspector; -import java.beans.PropertyDescriptor; -import java.util.Date; - -import org.apache.camel.Exchange; -import org.apache.camel.Processor; -import org.hibernate.search.annotations.Field; -import org.infinispan.Cache; -import org.infinispan.manager.CacheContainer; -import org.infinispan.query.Search; -import org.infinispan.query.dsl.FilterConditionContext; -import org.infinispan.query.dsl.Query; -import org.infinispan.query.dsl.QueryBuilder; -import org.infinispan.query.dsl.QueryFactory; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - - -public class QueryProcessor implements Processor { - - final Logger logger = LoggerFactory.getLogger(QueryProcessor.class); - - //cache container to be injected with another spring bean - private CacheContainer cacheContainer; - - - public CacheContainer getCacheContainer() { - return cacheContainer; - } - - - public void setCacheContainer(CacheContainer cacheContainer) { - this.cacheContainer = cacheContainer; - } - - @Override - public void process(Exchange ex) throws Exception { - - //Get the targeted cachename from the exchange header - CacheDeclare the processor, service end point and route in the Camel context
- -{% highlight xml %} -Test the project
- -Run the Fuse project on your dev machine
- -mvn clean package camel:run- -
Insert a few entries by running a curl command
- -
-curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
- "uid": "1",
- "timestmp": "2017-04-07T19:30:00.000Z",
- "name": "start",
- "content": "party started" }' 'http://localhost:7123/event/1'
-
-curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
- "uid": "2",
- "timestmp": "2017-04-07T22:15:00.000Z",
- "name": "incident",
- "content": "police arrived" }' 'http://localhost:7123/event/2'
-
-curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
- "uid": "3",
- "timestmp": "2017-04-07T23:18:00.000Z",
- "name": "incident",
- "content": "host arrested" }' 'http://localhost:7123/event/3'
-
-curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
- "uid": "4",
- "timestmp": "2017-04-07T23:20:00.000Z",
- "name": "end",
- "content": "party ended" }' 'http://localhost:7123/event/4'
-
-
-List all events through a query without parameters
-curl -X GET --header 'Accept: application/json' 'http://localhost:7123/query/event/techlab.model.Event'-{% highlight json %} -[ { - "uid" : "1", - "timestmp" : 1491593400000, - "name" : "start", - "content" : "party started" -}, { - "uid" : "2", - "timestmp" : 1491603300000, - "name" : "incident", - "content" : "police arrived" -}, { - "uid" : "3", - "timestmp" : 1491607080000, - "name" : "incident", - "content" : "host arrested" -}, { - "uid" : "4", - "timestmp" : 1491607200000, - "name" : "end", - "content" : "party ended" -} -{% endhighlight %} - -
List all incidents through a query with a parameter
-curl -X GET --header 'Accept: application/json' 'http://localhost:7123/query/event/techlab.model.Event?name=incident'-{% highlight json %} -[ { - "uid" : "2", - "timestmp" : 1491603300000, - "name" : "incident", - "content" : "police arrived" -}, { - "uid" : "3", - "timestmp" : 1491607080000, - "name" : "incident", - "content" : "host arrested" -} ] -{% endhighlight %} - -
List all incidents at a certain hour with 2 parameters (the resolution of the indexed date field is by hour)
-{% highlight shell %} -curl -X GET --header 'Accept: application/json' 'http://localhost:7123/query/event/techlab.model.Event?name=incident×tmp=1491607000000' -{% endhighlight %} -{% highlight json %} -[ { - "uid" : "3", - "timestmp" : 1491607080000, - "name" : "incident", - "content" : "host arrested" -} ] -{% endhighlight %} - - - -Alternatively you can also use swagger-ui to test the services
--
-
- To test the services you can use swagger ui to explore the rest services. - Download swagger ui : https://github.com/swagger-api/swagger-ui/archive/v2.2.10.zip - -
- Unzip the package and open dist/index.html in your favorite browser -
- Enter the url of the api-doc : http://localhost:7123/api-doc -
-
-
--Deploy Fuse project to Fuse Server (Karaf) -
- --features:install camel-swagger-java camel-netty4-http camel-jackson -features:addurl mvn:org.apache.camel/camel-jbossdatagrid/6.5.1.Final-redhat-1/xml/features -features:install camel-jbossdatagrid -features:addurl mvn:org.infinispan/infinispan-embedded/6.3.1.Final-redhat-1/xml/features -features:install infinispan-embedded -features:addurl mvn:org.infinispan/infinispan-remote/6.3.1.Final-redhat-1/xml/features -features:install infinispan-remote -- -
-Setting up a cache store to persist data -
diff --git a/_posts/2017-04-07-fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries.html b/_posts/2017-04-07-fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries.html deleted file mode 100644 index 91e4478b..00000000 --- a/_posts/2017-04-07-fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries.html +++ /dev/null @@ -1,716 +0,0 @@ ---- -layout: default -title: "Fuse Data Grid Integration via Hot Rod with Persistence and Indexing for Advanced Queries" -date: 2017-04-07 01:00:00 +0200 -published: 2017-04-07 01:00:00 +0200 -comments: true -categories: development -tags: [jboss-fuse, data-grid, hot-rod, protobuf, lucene] -github: "https://github.com/alainpham/techlab-fuse-jdg-simple-hotrod" ---- - --Building Integration and Services platform with JBoss Fuse is great. -It is even better when you add a distributed in memory data base solution such as JBoss Data Grid to the mix. -This article will show how to make both technologies work together using the camel-jbossdatagrid component. -We will go through the setup of a JBoss Data Grid server with persistence and see how to use it in a JBoss Fuse application through the remote Hot Rod client. -Furthermore we will see how to take advantage of Protocol buffers and Lucene to index data and perform content based queries. -
- - -In the example, we will buil rest services to store and retrieve some business events.
- -
-
-In summary the steps in this tutorial are the following :
- --
-
- Create an empty Fuse project in JBoss Developer Studio with Spring DSL -
- Add necessary dependencies to the pom.xml file -
- On the Data Grid server, configure a cache called "event" with an index (Lucene) and a cache store (H2 database as an example) -
- Launch an instance of H2 database and Data Grid Server -
- Create an POJO as a model of an event -
- Annotate the POJO with protobuf annotations to specify which fields should be indexed for queries -
- Create basic generic camel routes for getting and putting events into the cache -
- Create a query service with dynamic parameters -
- Test the services -
- Deploy the service on a Fuse Server -
Prerequisites
--
-
- JDK 1.8+ installed
- http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html - -
- - Maven 3.3+ installed
-https://maven.apache.org/download.cgi -
-
Versions used
--
-
- JBoss Fuse 6.3.x
-https://developers.redhat.com/products/fuse/download/ -
- - JBoss Developer Studio Integration Stack 10.1.0 (optional)
-https://developers.redhat.com/products/fuse/download/ -
- - JBoss Data Grid Server 6.5.x
- https://developers.redhat.com/products/datagrid/download/ -
- - Swagger UI (optional)
- https://github.com/swagger-api/swagger-ui/archive/v2.2.10.zip - -
-
-Access modes to JBoss Data Grid -
-JBoss Data Grid can be accessed through 2 different modes : Library Mode and Server Mode. -
-In Server Mode, Data Grid is installed as a distant server and Fuse gets access to the objects through the Hot Rod client. -
--In library mode (or embedded), the Fuse engine uses it's own JVM memory to contain objects that is part of the Data Grid. -In other words the Fuse engine acts as if it was a node amongst the Data Grid cluster. -This mode gives access to advanced features such as transactions and locking and is actually quite easy to setup. -The down side is that it is not possible to perform queries with the infinispan-embedded-query library with Fuse in a Karaf container. -The main reason is that infinispan-embedded-query includes full dependencies to libraries such as hibernate. Hence it is not suited for an OSGI container such as Karaf. -
--In this article we will explore the possibilities of the Server Mode with the Hot Rod client. The Library Mode will be discussed in an other article. -
- --Create a Fuse project -
--
-
- Start by creating a Fuse project with a Spring Camel Context in JBoss Developer Studio - (Alternatively it can also be created with a maven archetype via command line) -
- In JBoss Developer Studio go to files -> New -> Fuse Integration Project -
- Name the project techlab-fuse-jdg-simple-hotrod -
- Select runtime version 2.17.0.redhat-630224 -
- Start an empty project with Spring DSL -
- Click finish -
-
-
-
-
-Edit information and dependencies in pom.xml file -
--
-
- Change group id, artifact id to identify the project -
- Add dependency to camel-jbossdatagrid component : this enables invocation of Data Grid endpoints in a Camel route
-Note that the version should match the version of the Data Grid Server. -
- - Add dependency to h2 : as an example of cache store we are using an H2 database -
- Add dependency to camel-netty4-http, camel-swagger-java, camel-jackson : will be used to expose rest services to access the objects stored in the cache -
Cache Container configuration on Data Grid Server
--
-
- In the Data Grid installation folder, open the file standalone/configuration/standalone.xml -
- In the xml file, declare a cache named "event" -
- Configure a persistence cache store so that events can survive a server reboot. As an example here we will use an H2 Database. -
- Activate indexing to enable advanced queries using the Lucene engine -
- Configure eviction so that objects get unloaded from memory when not used. This is to avoid running out of memory. - Evicted entries stay in the cache store and can be reloaded when needed -
Launch H2 data base
- -Download H2 data base here :
- https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/h2database/h2-2012-07-13.zip
-
-
Run the database with the following parameters
- -{% highlight shell %} -java -cp h2*.jar org.h2.tools.Server -tcp -tcpAllowOthers -tcpPort 8942 -baseDir ./h2dbstore -web -webAllowOthers -webPort 11112 -{% endhighlight %} - -Now we can start the JBoss Data Grid Server by going to its installation folder and running :
-bin/standalone.sh- -
Create a model class for events and annotate for indexing
- -Create a class and annotate it so that it is indexed by Data Grid. To enable indexing through Hot Rod, POJOs need to serialized as protocol buffers
- -{% highlight java %} -package techlab.model; - -import java.io.Serializable; -import java.util.Date; - -import org.infinispan.protostream.annotations.ProtoDoc; -import org.infinispan.protostream.annotations.ProtoField; - -@ProtoDoc("@Indexed") -public class Event implements Serializable{ - private static final long serialVersionUID = 1L; - private String uid; - private Date timestmp; - private String name; - private String content; - - @ProtoField(number = 1) - public String getUid() { - return uid; - } - - public void setUid(String uid) { - this.uid = uid; - } - - @ProtoDoc("@IndexedField(index = true, store = false)") - @ProtoField(number = 2) - public Date getTimestmp() { - return timestmp; - } - - public void setTimestmp(Date timestmp) { - this.timestmp = timestmp; - } - - @ProtoDoc("@IndexedField(index = true, store = false)") - @ProtoField(number = 3) - public String getName() { - return name; - } - public void setName(String name) { - this.name = name; - } - - @ProtoField(number = 4) - public String getContent() { - return content; - } - - public void setContent(String content) { - this.content = content; - } -} -{% endhighlight %} - -Create rest services to get and put entries into the cache
- -Write a CacheManager factory. - Note that protobuf schemas can be generated from the annotated class and then registered to the Data Grid Server on the reserved in metadata cache
- -{% highlight java %} -package techlab.factory; - -import java.io.IOException; - -import org.infinispan.client.hotrod.RemoteCache; -import org.infinispan.client.hotrod.RemoteCacheManager; -import org.infinispan.client.hotrod.configuration.ConfigurationBuilder; -import org.infinispan.client.hotrod.marshall.ProtoStreamMarshaller; -import org.infinispan.protostream.SerializationContext; -import org.infinispan.protostream.annotations.ProtoSchemaBuilder; -import org.infinispan.protostream.annotations.ProtoSchemaBuilderException; -import org.infinispan.query.remote.client.ProtobufMetadataManagerConstants; - -import techlab.model.Event; - -public class RemoteCacheManagerFactory { - - ConfigurationBuilder clientBuilder; - - public RemoteCacheManagerFactory(String hostname, int port) { - clientBuilder = new ConfigurationBuilder(); - clientBuilder.addServer() - .host(hostname) - .port(port) - .marshaller(new ProtoStreamMarshaller()); - } - - public RemoteCacheManager newRemoteCacheManager() throws ProtoSchemaBuilderException, IOException { - RemoteCacheManager remoteCacheManager = new RemoteCacheManager(clientBuilder.build()); - - SerializationContext ctx = ProtoStreamMarshaller.getSerializationContext(remoteCacheManager); - - ProtoSchemaBuilder protoSchemaBuilder = new ProtoSchemaBuilder(); - - //create a protobuf schema file from the annotated class. Protobuf marshallers and unmarshallers are generated automtically - String eventSchema = protoSchemaBuilder - .fileName("event.proto") - .packageName("techlab") - .addClass(Event.class) - .build(ctx); - - //register the protobuf schema in the remote cache - RemoteCacheConfigure the factory in the spring context
-{% highlight xml %} -Add a reusable endpoint to the Camel context
- -{% highlight xml %} -Use the rest DSL to create routes and expose services to do the basic operations
- -{% highlight xml %} -Create a query service with dynamic parameters
- -Define a rest service that allows to pass any http query parameters
-
(i.e http://localhost:7123/query/event/techlab.model.Event?timestmp=1462208399999&name=ended)
Create classes that generate Data Grid Queries. Note that these classes are pretty generic as they use reflection and are suitable to any data model. -
- -{% highlight java %} -package techlab.dg; - -import java.beans.BeanInfo; -import java.beans.IntrospectionException; -import java.beans.Introspector; -import java.beans.PropertyDescriptor; -import java.util.Date; -import java.util.Map; - -import org.apache.camel.component.infinispan.InfinispanQueryBuilder; -import org.infinispan.query.dsl.FilterConditionContext; -import org.infinispan.query.dsl.Query; -import org.infinispan.query.dsl.QueryBuilder; -import org.infinispan.query.dsl.QueryFactory; - -public class GenericQuery implements InfinispanQueryBuilder { - - private MapDeclare the beans, service endpoint and route in the Camel context
- -{% highlight xml %} -Test the project
- -Run the Fuse project on your developer machine
- -mvn clean package camel:run- -
Insert a few entries by running a curl command
- -
-curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
- "uid": "1",
- "timestmp": "2017-04-07T19:30:00.000Z",
- "name": "start",
- "content": "party started" }' 'http://localhost:7123/event/1'
-
-curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
- "uid": "2",
- "timestmp": "2017-04-07T22:15:00.000Z",
- "name": "incident",
- "content": "police arrived" }' 'http://localhost:7123/event/2'
-
-curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
- "uid": "3",
- "timestmp": "2017-04-07T23:18:00.000Z",
- "name": "incident",
- "content": "host arrested" }' 'http://localhost:7123/event/3'
-
-curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
- "uid": "4",
- "timestmp": "2017-04-07T23:20:00.000Z",
- "name": "end",
- "content": "party ended" }' 'http://localhost:7123/event/4'
-
-
-List all events through a query without parameters
-curl -X GET --header 'Accept: application/json' 'http://localhost:7123/query/event/techlab.model.Event'-{% highlight json %} -[ { - "uid" : "1", - "timestmp" : 1491593400000, - "name" : "start", - "content" : "party started" -}, { - "uid" : "2", - "timestmp" : 1491603300000, - "name" : "incident", - "content" : "police arrived" -}, { - "uid" : "3", - "timestmp" : 1491607080000, - "name" : "incident", - "content" : "host arrested" -}, { - "uid" : "4", - "timestmp" : 1491607200000, - "name" : "end", - "content" : "party ended" -} -{% endhighlight %} - -
List all incidents through a query with a parameter
-curl -X GET --header 'Accept: application/json' 'http://localhost:7123/query/event/techlab.model.Event?name=incident'-{% highlight json %} -[ { - "uid" : "2", - "timestmp" : 1491603300000, - "name" : "incident", - "content" : "police arrived" -}, { - "uid" : "3", - "timestmp" : 1491607080000, - "name" : "incident", - "content" : "host arrested" -} ] -{% endhighlight %} - -
List all incidents at a certain hour with 2 parameters
-{% highlight shell %} -curl -X GET --header 'Accept: application/json' 'http://localhost:7123/query/event/techlab.model.Event?name=incident×tmp=1491607080000' -{% endhighlight %} -{% highlight json %} -[ { - "uid" : "3", - "timestmp" : 1491607080000, - "name" : "incident", - "content" : "host arrested" -} ] -{% endhighlight %} - -Alternatively you can also use swagger-ui to test the services
--
-
- To test the services you can use swagger-ui to explore the rest services. - Download swagger-ui : https://github.com/swagger-api/swagger-ui/archive/v2.2.10.zip - -
- Unzip the package and open dist/index.html in your favorite browser -
- Enter the url of the api-doc : http://localhost:7123/api-doc -
-
-
--Deploy Fuse project to Fuse Server (Karaf) -
- -In the pom.xml file of the Fuse project, add dynamic import block to the maven-bundle-plugin.
-{% highlight xml%} -Generate bundle for deployment
- -mvn clean package- -
Connect to Fuse console and run these commands to install required dependencies
--features:install camel-swagger-java camel-netty4-http camel-jackson -features:addurl mvn:org.apache.camel/camel-jbossdatagrid/6.5.1.Final-redhat-1/xml/features -features:install camel-jbossdatagrid --
Install our Fuse project bundle
-{% highlight shell%} -osgi:install -s file:That's it, now we have our running Data Grid with persistence, indexes and we are able to access it in a Fuse Project. -
- -Thanks for reading
diff --git a/_posts/2017-04-11-openshift-quickstart-in-5-minutes.html b/_posts/2017-04-11-openshift-quickstart-in-5-minutes.html deleted file mode 100644 index 59c8b711..00000000 --- a/_posts/2017-04-11-openshift-quickstart-in-5-minutes.html +++ /dev/null @@ -1,173 +0,0 @@ ---- -layout: default -title: "Openshift Quickstart in 5 minutes" -date: 2017-04-11 01:00:00 +0200 -published: 2017-04-11 01:00:00 +0200 -comments: true -categories: development -tags: [openshift, paas, spring-boot] -github: "https://github.com/alainpham/rest-service" ---- - --Openshift is a PaaS solution based on Docker and Kubernetes. This article will show you how to install Openshift Origin in less than 5 minutes and deploy your first -Java application to it. -
- - - -Versions used
--
-
- Centos 7 -
- Openshift Origin Client v1.4.1 -
Install docker
- -{% highlight shell %} -sudo yum upgrade -sudo yum install docker - -{% endhighlight%} - -Edit /etc/sysconfig/docker and add these options
- -{% highlight shell %} -OPTIONS='--selinux-enabled --insecure-registry 172.30.0.0/16' -{% endhighlight%} - - - -Execute docker without sudo
-Replace the ${USER} by your user name
-{% highlight shell %} -sudo groupadd docker -sudo gpasswd -a ${USER} docker -{% endhighlight%} -Reboot your machine
- -Test docker
- -{% highlight shell %} -sudo systemctl enable docker.service -sudo systemctl start docker.service -docker run hello-world -{% endhighlight%} -Expected result :
--Hello from Docker! -This message shows that your installation appears to be working correctly. -.... -- -
Download Openshift client and install
-https://www.openshift.org/download.html#oc-platforms -Place the oc executable in the $PATH
--[demo@demo-centos-vm ~]$ oc version -oc v1.4.1+3f9807a -kubernetes v1.4.0+776c994 -- -
Launch Openshift Cluster
-{% highlight shell %} -oc cluster up -{% endhighlight%} - -{% highlight shell %} --- Checking OpenShift client ... OK -... - OpenShift server started. - The server is accessible via web console at: - https://YOUR_IP_ADDRESS:8443 - - You are logged in as: - User: developer - Password: developer - - To login as administrator: - oc login -u system:admin - - -{% endhighlight%} - -Open browser and to to https://YOUR_IP_ADDRESS:8443/ and login with user/pwd : developer/developer
- -
-
-Deploy sample application
-In a shell log in as admin and add the Java S2I (source to image) image streams to the openshift project. - This is important as we will use these images to build images from our own project source code
-{% highlight shell %} -oc login -u system:admin -oc create -n openshift -f https://raw.githubusercontent.com/fabric8io-images/s2i/master/image-streams.json -{% endhighlight%} - -Login as developer and add the application template of an app. You can customize these template as you wish. - This is just an example of a Spring Boot java application that exposes a rest web service. -
-If you are interested in how to create a Spring Boot Application go to this article : quickstart-rest-microservice-with-spring-boot
- -In short the template describes
--
-
- The BuildConfig : what base image needs to be used to build the images container here it's "fuse-java" (imported from the image stream previously) -
- The DeploymentConfig : which ports are exposed by each pod (container) -
- The Services: what services (tcp mostly) are exposed by the app. This is to have location independency to access services exposed by multiple pods (containers) -
- The Routes: in order to access a service from outside of Openshift, routes need to be created. -
Open browser and login as developer to your console https://YOUR_IP_ADDRESS:8443/
-Go to My Project and Click on add to project
-Add to project from template and select the previously added rest-service.
- -
-The template allows you ton change configurations before creating the different objects on the openshift cluster (i.e. BuildConfig, DeploymentConfig, Services, Routes ..)
--The source is cloned from the git repository and built using the Source2Image method. - Note that this is a Java project and all dependencies for the maven build needs to be downloaded. If you have an existing nexus or other maven repo you can configure it in the Maven Mirror Url. -This could accelerate build time. -
-
-
-Click on create and wait for the build and deployments to occur.
- -
-
-
-Test by calling service.
-curl http://YOUR_ROUTE_DOMAIN_NAME/svc/person/1-
Expected result
-{% highlight json %} -{"id":1,"name":"John Doe"} -{% endhighlight%} - -You can now also access infos about your running pods and access JMX methods and data through the Java Console
- -
-
- 
-
-Thanks for reading
- -Other resources
--
-
- Cloud Happiness - OpenShift Container Platform v3.4 install demo updated -
- Have your own Openshift Container Platform for development : Red Hat Container Development Kit -
- On March 07, 2019 Red Hat & the JBoss Community announced Quarkus. It is a Java framework that enables ultra low - boot - times and tiny - memory footprint for applications and services. It does that by taking advantage of GraalVM's native compilation - capabilities to produce an executable out of your Java code. Why is this considered to be a game - changer ? - In this article I will give my view and first impressions on this framework as someone who spends - a bit of time writing applications and services. I will get my hands dirty by building a - working Quarkus + Apache Camel example. -
- - -What's the big deal about Quarkus?
- -When Quarkus was announced 1 week ago, many have commented that this is a game changer for the future of - Java Application - Development. I must say that I have rarely been so excited about a project as I am right now for Quarkus ! - So what's the big deal here ? -
- -The official website quarkus.io states that Quarkus can be qualified as "Supersonic Subatomic Java". It is a "A - Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpot, crafted from the best of breed Java libraries - and standards". -
- -The existing Java ecosystem
-- To understand the huge value that Quarkus brings, we have to take a step back and get in the shoes of a typical - Enterprise Dev Team. - Java has been so broadly adopted that it has become the default technology for application - development in - many organizations. - Throughout the years a huge ecosystem of tools and libraries flourished around Java to solve all kinds - of challenges and problems. -
- -The adoption of containers combined with Java
- -- The recent adoption of Containers and Kubernetes has lead to a wave of - innovation around workload orchestration and scheduling. - Still, organizations wanted to keep leveraging the expertise around Java their developers have acquired. - It became a natural practice to put Java applications and micro services into containers to get the best of both - worlds. Organizations wanted a rich ecosystem - of libraries and all the innovations around container orchestration at the same time. -
- -
- 
-
-
The paradigm shift needed for Java to exist in the world of Containers
- - -- When a micro service needs to be scaled up and down quickly is when Java shows some weakness. - In fact Java has not been designed for being a short lived process that might be scheduled anywhere on a - distributed infrastructure. - It has rather been designed and optimized to run application servers for a long time and taking up all the - resources - available on a VM or a bare metal server. - The boot time was not such a critical constraint. In usual Java application the boot time of 4 to 10 seconds - (and even more) was - entirely acceptable. The initial memory footprint is usually also negligible due to the fact that a large amount of memory - would be dedicated to the application server anyway. -
- -- The paradigm has shifted in the world of containers and Kubernetes. The boot time has become highly critical. A low boot time - is what would allow a service to scale up and down quickly and achieve higher - workload density on cloud infrastructures. A minimal initial memory footprint per instance is what would - allow the service to start small and be scaled efficiently when needed. - In the extreme case, when considering the rise of Functions as a Service booting up quickly becomes even more critical. - In this case we aim at scaling the service down to zero instances when there are no requests and starting - it only when client requests are coming in. - We can see here that the boot time needs to be several orders of magnitude lower than what we are used to. -
- -
- 
-
Quarkus to the rescue
-- Given this context, we can now better understand the value that Quarkus brings. - In short, it's a framework that enables dev teams to use widely adopted Java libraries to build applications that - have an extreme low boot time and low initial memory footprint. - Quarkus takes advantage of GraalVM to compile java projects into a native executable that can just run. -
- -Get started : build a hello world
-Now lets go ahead and build our first hello world to see how it performs.
-- Create a simple project using the quarkus-maven-plugin -
- -{% highlight shell %} -mvn io.quarkus:quarkus-maven-plugin:0.11.0:create \ --DprojectGroupId=techlab \ --DprojectArtifactId=quarkus-hello-world \ --DclassName="techlab.GreetingResource" \ --Dpath="/hello" -{% endhighlight %} - -- This is the structure of the project : -
- -{% highlight shell %} -|-- pom.xml -|-- quarkus.log -|-- src -| |-- main -| | |-- docker -| | | `-- Dockerfile -| | |-- java -| | | `-- techlab -| | | `-- GreetingResource.java -| | `-- resources -| | `-- META-INF -| | |-- microprofile-config.properties -| | `-- resources -| | `-- index.html -{% endhighlight %} - -- The dependencies are pretty minimalist and clean since most of it is taken care of by the quarkus bom and plugins. -
- -{% highlight xml %} -- The plugin creates a simple project with a rest service operation returning hello as you can see in the file - GreetingResource.java. - It is pretty much the only thing that needs to be done here to have a running service. Quarkus is rather an - opinionated framework that aims at accelerating development using configuration files and annotations. -
- -{% highlight java %} -package techlab; - -import javax.ws.rs.GET; -import javax.ws.rs.Path; -import javax.ws.rs.Produces; -import javax.ws.rs.core.MediaType; - -@Path("/hello") -public class GreetingResource { - -@GET -@Produces(MediaType.TEXT_PLAIN) -public String hello() { - return "hello"; - } -} -{% endhighlight %} - -Let's run and test the project
- -- Now lets go ahead and run this project to see how fast it actually boots up. -
- -{% highlight shell %} -mvn compile quarkus:dev -{% endhighlight %} - - -{% highlight shell %} -[apham@aplinux quarkus-hello-world]$ mvn compile quarkus:dev -[INFO] Scanning for projects... -[INFO] -[INFO] ------------------------------------------------------------------------ -[INFO] Building quarkus-hello-world 1.0-SNAPSHOT -[INFO] ------------------------------------------------------------------------ -[INFO] -[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ quarkus-hello-world --- -[INFO] Using 'UTF-8' encoding to copy filtered resources. -[INFO] Copying 2 resources -[INFO] -[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ quarkus-hello-world --- -[INFO] Nothing to compile - all classes are up to date -[INFO] -[INFO] --- quarkus-maven-plugin:0.11.0:dev (default-cli) @ quarkus-hello-world --- -[INFO] Using servlet resources /home/workdrive/TAZONE/WORKSPACES/ws-quarkus/quarkus-hello-world/src/main/resources/META-INF/resources -Listening for transport dt_socket at address: 5005 -2019-03-17 23:10:29,640 INFO [io.qua.dep.QuarkusAugmentor] (main) Beginning quarkus augmentation -2019-03-17 23:10:29,980 INFO [io.qua.dep.QuarkusAugmentor] (main) Quarkus augmentation completed in 340ms -2019-03-17 23:10:30,155 INFO [io.quarkus] (main) Quarkus 0.11.0 started in 0.602s. Listening on: http://127.0.0.1:8080 -2019-03-17 23:10:30,156 INFO [io.quarkus] (main) Installed features: [cdi, resteasy] -{% endhighlight %} - -- We should bare in mind that here we are using the OpenJDK HotSpot and are not yet compiling our project to a native executable. - The results are already pretty remarkable as I could observe a boot time of 602 milliseconds. -
-(Edit Sept 17, 2017, prior to this date all my time conversions from seconds to milliseconds have been wrong. Thank you Faris for pointing it out :) )
-- Compile to native executable and run -
- -- In order to compile the project we need to setup GraalVM. - Download the VM from the following website http://www.graalvm.org/downloads/. - Extract the package somewhere and set the env variable GRAALVM_HOME to point to the root of its folder. -
-- Now we can run the following command to build a native executable. -
- -{% highlight shell %} -mvn package -Pnative -{% endhighlight %} - -- We run the native executable with the following command. -
- -{% highlight shell %} -./target/quarkus-hello-world-1.0-SNAPSHOT-runner -{% endhighlight %} - -{% highlight shell %} -[apham@aplinux quarkus-hello-world]$ ./target/quarkus-hello-world-1.0-SNAPSHOT-runner -2019-03-17 23:34:04,690 INFO [io.quarkus] (main) Quarkus 0.11.0 started in 0.004s. Listening on: http://127.0.0.1:8080 -2019-03-17 23:34:04,691 INFO [io.quarkus] (main) Installed features: [cdi, resteasy] -{% endhighlight %} - -- We can observe here that the boot time is enhanced by a significant order of magnitude which is 4 milliseconds. -
- -(Edit Sept 17, 2017, prior to this date all my time conversions from seconds to milliseconds have been wrong.)
- - -Enhanced Developer Experience
-- Now the good surprises don't end there. Since the boot times are so good, it also solve a big challenge for development workflow. - In fact, Quarkus also implements a livereload mechanism to reflect the changes performed on the code instantaneously. - Try to change some of the code in the project, hit Ctrl+s and we can observe how fast the service reloads to take the changes into account. - This is a big leap for developer productivity. As we can see here it took only 386 milliseconds to reload. -
-(Edit Sept 17, 2017, prior to this date all my time conversions from seconds to milliseconds have been wrong.)
-{% highlight shell %} -2019-03-18 09:59:58,988 INFO [io.qua.dev] (XNIO-1 task-1) Changes source files detected, recompiling [/home/workdrive/TAZONE/WORKSPACES/ws-quarkus/quarkus-camel-first-impressions/src/main/java/techlab/GreetingResource.java] -2019-03-18 09:59:59,274 INFO [io.quarkus] (XNIO-1 task-1) Quarkus stopped in 0.001s -2019-03-18 09:59:59,275 INFO [io.qua.dep.QuarkusAugmentor] (XNIO-1 task-1) Beginning quarkus augmentation -2019-03-18 09:59:59,366 INFO [io.qua.dep.QuarkusAugmentor] (XNIO-1 task-1) Quarkus augmentation completed in 91ms -2019-03-18 09:59:59,373 INFO [io.quarkus] (XNIO-1 task-1) Quarkus 0.11.0 started in 0.098s. Listening on: http://127.0.0.1:8080 -2019-03-18 09:59:59,373 INFO [io.quarkus] (XNIO-1 task-1) Installed features: [cdi, resteasy] -2019-03-18 09:59:59,373 INFO [io.qua.dev] (XNIO-1 task-1) Hot replace total time: 0.386s -{% endhighlight %} - -Let's add Camel to the mix
-Now that we have our hello world running. Let's try to add Camel to the mix to enable reusable integration patterns - and connectors in our projects. Quarkus already offers a variety extensions and Camel is one of those. At the time of writing - there are just a few extensions available. We won't get the whole set of 200+ Camel components compiled to native executable packages, - but I'm quite sure that the Quarkus ecosystem will grow quickly. -
- -- Let's list the available extensions. -
- -mvn quarkus:list-extensions-
- Then let's add the camel extension to it. -
-mvn quarkus:add-extension -Dextensions=io.quarkus:quarkus-camel-core- -
- As a result you should see the dependency to the quarkus camel extension being added to the pom file -
- -{% highlight xml %} -- Lets create a simple class containing a Camel route to see if things work. Add a new class called CamelRouteBuilder to the source folder. - With the following content. -
- -{% highlight java %} -package techlab; - -import org.apache.camel.builder.RouteBuilder; - -public class CamelRouteBuilder extends RouteBuilder { - - public void configure() { - from("timer://testTimer").log("test"); - } -} -{% endhighlight %} - -At the time of writing there is no documentation on how to get started with camel in Quarkus. - I found an error just trying to run this class complaining about properties not being available.
-Property with key [camel.defer] not found- -
Searching through the source code I found that it is looking some properties in the application.properties file. - The one file of interest can be found here -
- -Update July 4, 2019 : As of today, version 0.18 is out now the camel core plugin has changed and does not require to add those properties anymore
- -
- So lets go ahead and create a file named application.properties in the folder src/main/resources with the following content -
- --camel.defer=false -camel.conf= -camel.confd= -camel.routesUri= -camel.dump=false -- -
Now we can run our project with
mvn compile quarkus:dev- -{% highlight java %} -2019-03-18 10:43:27,702 INFO [io.qua.cam.cor.run.FastCamelContext] (main) Route: route1 started and consuming from: timer://testTimer -2019-03-18 10:43:27,771 INFO [io.quarkus] (main) Quarkus 0.11.0 started in 1.509s. Listening on: http://127.0.0.1:8080 -2019-03-18 10:43:27,771 INFO [io.quarkus] (main) Installed features: [camel-core, cdi, resteasy] -2019-03-18 10:43:28,720 INFO [route1] (Camel (camel-1) thread #1 - timer://testTimer) test -2019-03-18 10:43:29,703 INFO [route1] (Camel (camel-1) thread #1 - timer://testTimer) test -{% endhighlight %} -
- We can note that the boot time is a bit higher when including camel, it is now 1.509s. -
- -Let's compile it to a native executable and see how it behaves.
- --mvn package -Pnative -/target/quarkus-camel-first-impressions-1.0-SNAPSHOT-runner -- -{% highlight shell %} -2019-03-18 11:05:57,835 INFO [io.qua.cam.cor.run.FastCamelContext] (main) Route: route1 started and consuming from: timer://testTimer -2019-03-18 11:05:57,838 INFO [io.quarkus] (main) Quarkus 0.11.0 started in 0.007s. Listening on: http://127.0.0.1:8080 -2019-03-18 11:05:57,838 INFO [io.quarkus] (main) Installed features: [camel-core, cdi, resteasy] -2019-03-18 11:05:58,835 INFO [route1] (Camel (camel-1) thread #0 - timer://testTimer) test -{% endhighlight %} - -
We can see that the boot time is of 7 milliseconds!
-(Edit Sept 17, 2017, prior to this date all my time conversions from seconds to milliseconds have been wrong.)
- -- In summary, why is Quarkus so amazing ? -
- --
-
- Lower boot times with significant orders of magnitude. -
- Puts Java Applications back into the game for efficient scale up in Cloud environments -
- Puts Java Applications back into the game for Functions as a Service, being able to scale down to zero and boot - up only when required -
- Amazing Developer experience and productivity through fast "Live Reload" capabilities -
Other resources
--
-
- - Quarkus guides - -
- Introducing - Quarkus: a next-generation Kubernetes native Java framework -
- Why Quarkus, blog article by Emmanuel Bernard -
- When using Apache Camel with Quarkus as of today, we are limited to a number of Camel connectors. An important one, being - the Apache Kafka connector. - The Kafka connector provided through the Smallrye Kafka Extension is available for Quarkus though. So in this - article, I will show how to - wire the Smallrye Kafka connector and Camel together. - We will also be using the camel-servlet component to reuse the undertow http endpoint provided by Quarkus. To - spice things up a little we will use the XML DSL of Camel. - All of this will be natively compiled in the end. -
- - -Setup a project
- -In a previous article I showed how to create a hello world using Camel and Quarkus. Here We will try to do something - more advanced. Lets start by setting up the project
- -{% highlight shell %} - -mkdir quarkus-kafka-camel-servlet - -cd quarkus-kafka-camel-servlet - -mvn io.quarkus:quarkus-maven-plugin:0.18.0:create \ --DprojectGroupId=agilabs \ --DprojectArtifactId=quarkus-kafka-camel-servlet \ --DclassName="agilabs.GreetingResource" \ --Dpath="/hello" -{% endhighlight %} - -- Add the following dependencies to the pom.xml file. -
-{% highlight xml %} -Configure Camel
- -Application properties
- -Configure application properties in src/main/resources/application.properties
- --quarkus.camel.servlet.url-patterns = /camel/* -%dev.quarkus.camel.routes-uris = camel/routes.xml -quarkus.camel.routes-uris=file:/camel/routes.xml -quarkus.camel.defer-init-phase = true -- -
- We will match all calls that go to /camel/* to the camel servlet component. In dev mode we will scan the file - routes.xml for camel routes located in the classpath, in the resource folder camel. -
-- When launched in "production mode" with the native executable the resources are not included by default in the - binary. I'm externalizing the routes.xml file to the folder /camel so that it can be set - and changed dynamically. This is actually quite interesting because we get to have a configurable native executable - that we can fine tune by switching the routes during run time. - For some when I tried to change the property quarkus.camel.routes-uris during native run time, it's new value doesn't - seem to be taken into account. So this file path is kind of like being hard coded right now.. -
-Edit July 6,2019 : I discovered that there are different Configuration Phases - : BUILD_TIME, BUILD_AND_RUN_TIME_FIXED & RUN_TIME. - It turns out this property is set to BUILD_AND_RUN_TIME_FIXED, so it is not dynamically read during native run time for the moment. - More details here. -
-- Quarkus.camel.defer-init-phase property is set to true because it seems that the camel context is created before all - the CDI bean wiring and dependency injection happens. - I need to do further digging to understand entirely how this works. -
- -- Camel Routes XML Files -
- -- Lets set up the routes in the src/main/resources/camel/routes.xml file. -
- -{% highlight xml %} - -The first route is to consume messages coming from Kafka. We will wire later on a camel producer template to send - messages to this route from any other class in application
-The second route listens to http requests coming on the path http://0.0.0.0:8080/camel/send. - It will use a bean that incorporates an emitter to send messages. -
- -Configure sinks for Kafka topics
- -Application properties
- --mp.messaging.incoming.in-events.connector=smallrye-kafka -mp.messaging.incoming.in-events.topic=events -mp.messaging.incoming.in-events.value.deserializer=org.apache.kafka.common.serialization.StringDeserializer - -mp.messaging.outgoing.out-events.connector=smallrye-kafka -mp.messaging.outgoing.out-events.topic=events -mp.messaging.outgoing.out-events.value.serializer=org.apache.kafka.common.serialization.StringSerializer -- -
We will configure a smallrye-kafka connector for incoming messages and outgoing messages
- -Create a consumer
- -Now lets go ahead and create a consumer with the @Incoming annotation. We will need to inject a Camel - ProducerTemplate instance into the consumer so that the message can be forwarded to our Camel direct endpoint
- -{% highlight java %} -package agilabs; -import javax.enterprise.context.ApplicationScoped; -import javax.inject.Inject; -import org.apache.camel.ProducerTemplate; -import org.eclipse.microprofile.reactive.messaging.Incoming; - -@ApplicationScoped -public class Consumer { - - @Inject - ProducerTemplate camelProducer; - - @Incoming("in-events") - public void consume(String message) { - camelProducer.sendBody("direct:receive-events", message); - } -} -{% endhighlight %} - -Create a producer
- -- Now lets create a producer. Note that this producer will be used by the camel xml context. So it needs to support reflection to find the different methods dynamically. - We need to register this class for reflection so that it can work once compiled to native code. -
- -{% highlight java %} -package agilabs; - -import javax.enterprise.context.ApplicationScoped; -import javax.inject.Inject; - -import org.apache.camel.ProducerTemplate; -import org.eclipse.microprofile.reactive.messaging.Incoming; - -import io.quarkus.runtime.annotations.RegisterForReflection; -import io.smallrye.reactive.messaging.annotations.Emitter; -import io.smallrye.reactive.messaging.annotations.Stream; - -@ApplicationScoped -@RegisterForReflection -public class Producer { - - @Inject - @Stream("out-events") - EmitterWiring it all together
- -Now that we have everything configured. We need to wire everything together. Register the SmallRye producer into the - camel context and inject a camel producer template in the Quarkus CDI context.
-As mentioned earlier, it seems that the Camel context is started earlier than the CDI context, so when Camel - tries to look for a bean called smallrye-producer it will fail to start because it can't find it. - This also comes from the fact that ApplicationScoped beans are instantiated in a lazy fashion. - Since the smallrye-producer is never called in the CDI context, it will never be initialized. - For these reasons we need to observe the Camel InitializedEvent and register the SmallRye producer at that specific time. -
-- The @Produces annotation provides the Camel producer template to the CDI context. -
- -{% highlight java %} -package agilabs; -import javax.enterprise.event.Observes; -import javax.enterprise.inject.Produces; -import javax.inject.Inject; -import org.apache.camel.CamelContext; -import org.apache.camel.ProducerTemplate; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import io.quarkus.camel.core.runtime.InitializedEvent; -import io.smallrye.reactive.messaging.annotations.Emitter; -import io.smallrye.reactive.messaging.annotations.Stream; - -public class CamelWiring{ - - @Inject - Producer smallRyeProducer; - - @Inject - CamelContext camelContext; - - public void registerSmallryeEmitter( @Observes InitializedEvent ev ) throws Exception { - camelContext.getRegistry().bind("smallrye-producer", smallRyeProducer); - } - - @Produces - public ProducerTemplate buildCamelProducerTemplate() throws Exception { - return camelContext.createProducerTemplate(); ; - } -} -{% endhighlight %} - -Compiling and testing the project
- -Now lets compile it to native code. The compilation is quite memory & CPU intensive. I guess this is due to the costly compilation of XML support libraries. - At peak I observed 9.8 GB of memory consumption. The whole compilation process took about 7 to 8 minutes
- --git clone https://github.com/alainpham/quarkus-kafka-camel-servlet.git -cd quarkus-kafka-camel-servlet -mvn package -Pnative -- -
- If you are in a rush. I have already built a container image for you. You can run everything wih only one command - using the docker-compose.yml file. It will boot a zookeeper and a kafka broker using images provided by the Strimzi project and - also a container with the current project. -
- --docker-compose up -- -
You can also get the container image directly here :
-
- https://hub.docker.com/r/alainpham/quarkus-kafka-camel-servlet
-
Test it!
--curl http://localhost:8080/camel/send -X POST -H "Content-Type: text/plain" -d 'Hello there!!' -- -
The output should be
- --quarkus-kafka-camel-servlet_1 | 2019-07-05 13:12:58,025 INFO [send-kafka-events] (executor-thread-1) Sending -quarkus-kafka-camel-servlet_1 | 2019-07-05 13:12:58,027 INFO [io.sma.rea.mes.kaf.KafkaSink] (vert.x-eventloop-thread-1) Message org.eclipse.microprofile.reactive.messaging.Message$$Lambda$5e1799c60041209b57937765b33186671a72f6f8@7f35f7425e98 sent successfully to Kafka topic 'events' -quarkus-kafka-camel-servlet_1 | 2019-07-05 13:12:58,027 INFO [receive-kafka-events] (vert.x-eventloop-thread-0) Received from kafka broker : Hello there! -- -
Thanks for reading !
- - -Resources I used for inspiration
--
-
- - Quarkus Kafka Guide - -
- - Quarkus - Contexts and Dependency Injection - -
- - Quarkus - Camel example by Guillaume Nodet - -
- Smallrye - Reactive Messaging Unit Test -
引言
+学了`Python`基础语法后想锻炼一下解决实际问题的水平，那本文介绍的小项目刚刚好。通过本文中项目，你可以学会不打开12306或者APP就能查询火车票，同时熟悉`Python`库的使用方法，了解`Python`程序设计思路。 + + + +这个项目来源于[此处](https://www.shiyanlou.com/courses/623)，本文是在学习后进行的总结，最终效果如图所示。 + +
+
+本项目需要`Python`基础语法知识，在项目中可以学习到如下内容：
+- `docopt`、`requests`、`colorama`和`prettytable`库的使用
+- 命令行安装需要的`Python`库
+- 分析网站的请求和响应过程
+
+## 设计思路
+
+为了实现如图所示功能效果，我们需要先进行需求分析。
+- 当我们查询火车票时一般来说需要指定条件，例如列车时间、起始站点等信息才能查看列车时刻信息，因此，我们需要明确定义有哪些查询条件；
+- 然后需要有命令行界面，用来接受用户输入的查询条件；
+- 根据用户输入的查询条件，去寻找列车时刻信息；
+- 最后将获得的列车时刻信息经过排版再显示出来。
+
+这样，总结下来可以分为如下几个大的模块
+- 命令行界面，用于接受用户输入的查询条件
+- 列车信息API，向12306服务器发起查询请求，并获得列车信息
+- 列车信息展示，美观的展示出查询到达列车信息
+
+## 知识点
+### 项目所需`Python`库
+
+- `docopt`库：终端命令行参数解析器，[查看详情](https://github.com/docopt/docopt);
+- `requests`库：简单易用的Python HTTP库，[快速上手文档](http://docs.python-requests.org/zh_CN/latest/user/quickstart.html);
+- `prettytable`库：格式化输出打印内容，[查看详情](https://github.com/jazzband/prettytable);
+- `colorama`库：命令行着色工具，[查看详情](https://pypi.org/project/colorama/);
+
+### 命令行一键安装
+
+通过如下命令一键安装以上资源：
+
+`pip3 install requests prettytable docopt colorama`
+
+## 分步讲解
+下面开始构建属于自己的命令行列车时刻查询工具。
+
+### 命令行界面
+
+```python
+# coding: utf-8
+
+"""命令行火车票查看器
+
+Usage:
+ tickets [-gdtkz] 
+
+### 获取列车信息
+
+查火车票只能去12306，但是铁道部没有提供API，那么我们就不能查询了吗，答案是否定的。我们现在在网站上查询几次列车信息，寻找接口的蛛丝马迹。
+
+#### 分析页面接口
+
+- 在列车查询页面，然后打开浏览器调试模式（Windows下Chrome浏览器快捷键是F12），进入`Network`中，查看`XHR`，然后查询列车信息
+
+ 
+
+- 点击箭头指向的文件，我们可以看到在 `Headers`栏目中的`Request URL`中包括了查询的所有条件信息：出发地、目的地、出发日
+
+ 
+
+- 再点击`Preview`查看返回结果，这个就是查询到的列车时刻信息了
+
+ 
+
+- 知道如何查询列车车次信息后，还有一个问题，就是我们输入的是中文车站名，接口处理的是英文车站代码，这个映射关系还不知道，需要继续查看页面加载的文件，寻找这个对应关系的文件，结果找到了如图文件
+
+ 
+
+#### 使用`requests`库
+
+接口信息分析完毕，下面用`requests`库进行实战。
+
+- 首先，来获得我们想要的车站英文代码数据，通过向车站中英文关系文件发起请求获得响应，分析结果
+
+```python
+# --coding: UTF-8--
+import re, requests
+from pprint import pprint
+
+# 车站中英文文件URL
+url = 'https://kyfw.12306.cn/otn/resources/js/framework/station_name.js?station_version=1.9098'
+
+# 使用requests库发起请求，获得响应的json文件
+resp = requests.get(url)
+
+# 对获得的响应文件使用正则表达式，获得车站中文名和英文代码组成的元祖构成的列表
+stations = re.findall(r'([\u4e00-\u9fa5]+)\|([A-Z]+)', resp.text)
+
+# 格式化输出车站
+pprint(dict(stations), indent=4)
+
+```
+
+ — `stations` 返回的结果如下图，是车站中文名和英文代码构成的元组组成的列表
+
+
+
+ — 使用 `python parse_stations.py > stations.py`命令并生成`stations.py`文件，车站文件内容为中文名和英文代码构成的字典，这样就可以导入`stations.py`文件后根据键（车站中文名）从字典中检索对应值（车站中文对应的英文代码）
+
+
+
+- 回顾一下，在命令行界面输入的起始车站中文名和列车出发日期，这些信息都在`arguments`这个字典对象中，只需要根据输入的信息匹配车站英文代码，然后访问车次查询接口，这样就可以查询列车信息了
+
+ — 修改命令行函数`cli`中代码
+
+ ```python
+ def cli():
+ """command-line interface"""
+ arguments = docopt(__doc__)
+ # 根据命令行用输入的起始车站名称、日期，查询stations中对应的英文代码
+ from_station = stations.get(arguments['
+
+#### 解析车次信息
+
+对获取的车次信息一头雾水，那么需要尝试继续在网站中寻找线索。一般来说，网站的js文件中会定义这些数据结构，并标注每个字段的含义，所以现在来寻找这样的js文件。
+
+进入浏览器调试模式，查看源代码`source`。猜测是图中所示的文件可能包括车票信息，因为查询车次就是查询列车余票信息。
+
+
+
+将压缩的js代码调整下格式，然后尝试搜索“硬座”与列车相关的信息关键词，结果找到了如下代码，这下知道坐席代码了：
+
+
+
+继续寻找“硬座”，发现并没有更多有价值内容，然后尝试搜索“硬座”的键值“YZ”，发现关键代码：
+
+
+
+完美，这段代码十分详细的介绍了车次信息中各个值的含义，我们再拿来和在12306查询到的车次信息比对发现完全正确，这样就可以解析车次信息了。
+
+### 列车信息展示
+
+#### 构建列车信息类
+
+接下来使用类来封装获取的列车信息，整体思路是，通过类来处理页面爬虫获取的列车信息，并将数据清洗后展示出来，列车信息类`TrainsInfo`的代码如下：
+
+```python
+class TrainsInfo:
+ def __init__(self, available_trains, from_to_stations, options):
+ """
+ 定义类用到的变量：列车信息，列车类型选项
+ available_trains：列车信息
+ fom_o_saions：起止车站，模糊匹配，例如杭州，杭州东等
+ options：列车类型选项
+ """
+ self.available_trains = available_trains
+ self.from_to_stations = from_to_stations
+ self.options = options
+
+ @property
+ def trains(self):
+ for each_train in self.available_trains:
+ each_train_split = each_train.split('|')
+ train_code = each_train_split[3]#车次
+
+ if not self.options or train_code[0].lower() in self.options):
+ """
+ 显示符合条件的列车信息，即列车类型匹配g/d/z等 或者
+ 不输入列车等级 也可查询
+ """
+ trains = [train_code, #车次
+ each_train_split[6], #起点车站代码
+ each_train_split[7], #终点车站代码
+
+ each_train_split[8], #发车时间
+ each_train_split[9], #到达时间
+ each_train_split[10], #历时
+
+ each_train_split[11], #网上购票
+ each_train_split[26] if each_train_split[26] else '--', #无座
+ each_train_split[27] if each_train_split[27] else '--', #硬座
+ each_train_split[24] if each_train_split[24] else '--', #软座
+ each_train_split[28] if each_train_split[28] else '--', #硬卧
+ each_train_split[33] if each_train_split[33] else '--', #动卧
+ each_train_split[23] if each_train_split[23] else '--', #软卧
+ each_train_split[21] if each_train_split[21] else '--', #高级软卧
+ each_train_split[30] if each_train_split[30] else '--', #二等座
+ each_train_split[31] if each_train_split[31] else '--', #一等座
+ each_train_split[25] if each_train_split[25] else '--', #特等座
+ each_train_split[32] if each_train_split[32] else '--', #商务座
+ each_train_split[22] if each_train_split[22] else '--', #其他
+ ]
+ yield trains
+
+
+ def print_train_info(self):
+ for train in self.trains:
+ print(train)
+```
+
+为了调用类，修改后的`cli`代码如下：
+
+```python
+def cli():
+ """command-line interface"""
+ arguments = docopt(__doc__)
+ from_station = stations.get(arguments['
+
+看一下12306的查询结果，比对一下，完全一致。
+
+
+
+#### 美化输出界面
+
+但是工作到这里并没有结束，这个输出界面太丑陋了，需要精致一些。使用prettytable库来将信息用表格形式输出，在调整些颜色即可。
+
+调整列车信息打印函数，通过表格输出信息，在使用前需要引用`prettytable`库 `import prettytable`
+
+```python
+ def print_train_info(self):
+ # 创建表格
+ tb = prettytable.PrettyTable()
+ # 设置表格标题
+ tb.field_names = self.headers
+ for train in self.trains:
+ # 增加每行数据
+ tb.add_row(train)
+ print(tb)
+
+```
+
+列车信息类中需要对train信息进行颜色标识，整体代码修改后如下：
+
+```python
+init()
+
+class TrainsInfo:
+ headers = '车次 车站 时间 历时 有票 无座 硬座 软座 硬卧 动卧 软卧 高级软卧 二等座 一等座 特等座 商务座 其他'.split()
+
+ def __init__(self, available_trains, from_to_stations, options):
+ """
+ 定义类用到的变量：列车信息，列车类型选项
+ available_trains：列车信息
+ fom_o_saions：起止车站，模糊匹配，例如杭州，杭州东等
+ options：列车类型选项
+ """
+ self.available_trains = available_trains
+ self.from_to_stations = from_to_stations
+ self.options = options
+
+ @property
+ def trains(self):
+ for each_train in self.available_trains:
+ each_train_split = each_train.split('|')
+ train_code = each_train_split[3]#车次
+
+ if not self.options or train_code[0].lower() in self.options:
+ """
+ 显示符合条件的列车信息，即列车类型匹配g/d/z等 或者
+ 不输入列车等级 也可查询
+ """
+ trains = [train_code, #车次
+ '\n'.join([Fore.GREEN + self.from_to_stations[each_train_split[6]] + Fore.RESET,
+ Fore.RED + self.from_to_stations[each_train_split[7]] + Fore.RESET]),
+
+ '\n'.join([Fore.GREEN + each_train_split[8] + Fore.RESET,
+ Fore.RED + each_train_split[9] + Fore.RESET]),
+
+ each_train_split[10], #历时
+
+ each_train_split[11], #网上购票
+ each_train_split[26] if each_train_split[26] else '--', #无座
+ each_train_split[27] if each_train_split[27] else '--', #硬座
+ each_train_split[24] if each_train_split[24] else '--', #软座
+ each_train_split[28] if each_train_split[28] else '--', #硬卧
+ each_train_split[33] if each_train_split[33] else '--', #动卧
+ each_train_split[23] if each_train_split[23] else '--', #软卧
+ each_train_split[21] if each_train_split[21] else '--', #高级软卧
+ each_train_split[30] if each_train_split[30] else '--', #二等座
+ each_train_split[31] if each_train_split[31] else '--', #一等座
+ each_train_split[25] if each_train_split[25] else '--', #特等座
+ each_train_split[32] if each_train_split[32] else '--', #商务座
+ each_train_split[22] if each_train_split[22] else '--', #其他
+ ]
+ yield trains
+```
+
+这里面用到了`colorama`命令行着色工具，使用前需要引入库 `from colorama import init, Fore`，然后初始化`init()` 并设置颜色`Fore.GREEN` `Fore.RED` `Fore.RESET`
+
+
+
+## 总结
+
+本项目最核心知识点在于`requests`库的使用，难点在于如何分析js文件寻找车站和英文关系对应以及列车车次信息获取后如何解析。
+
+其他的知识点还有`yield`使用，`@property`装饰器，`dicopt`命令行界面以及`prettytable`表格输出`colorama`表格上色。
+
+详细代码请查阅：
+
+
+Every linked list consists of nodes, we call the first node as 'head' and the last as 'tail', and for each node it has two components:
+- data, it allows a node in the linked list to **store an element** of data that can be of type string, character, number, or any other type of object.
+- next, it is **a pointer** that points from one node to another.
+
+Sometimes, we may need a dummy head in the linked list that can help us with intrusion or deletion the first node.
+
+### 2. Difference with Array
+
+#### Insertion/ Deletion
+
+- If we are given the exact pointer after which we have to insert another node or delete a node, it will be a constant-time operation.
+- However, the insertion/deletion operation is in `O(n)` operations for insertion/deletion of value at the array. Due to the shifting of the elements, the time complexity is `O(n)`.
+
+#### Accessing Elements
+
+- Accessing an n-th element in a linked list is an `O(n)` operation given that you have access to the head node of the linked list.
+- It is a constant time operation to access elements in arrays, if given an array and an index.
+
+### 3. Implementation
+
+#### 1). General classes
+Single linked lists contain two kinds of classes:
+- `Node` class, Every node is going to consist of `data` and `next`.
+- `LinkedList` class, except constructor, it initially contains `print_list()` methos to show each node's data.
+
+
+```python
+class Node:
+ def __init__(self, data):
+ self.data = data
+ self.next = None
+
+class LinkedList:
+ def __init__(self):
+ self.head = None
+
+ def print_list(self):
+ """
+ it will print out the data of each node from head to tail.
+ """
+ cur_node = self.head
+ while cur_node:
+ print(cur_node.data)
+ cur_node = cur_node.next
+
+```
+
+#### 2). Insertion
+
+Now we'll insert elements in a linked list by different ways:
+- `append`, the append method will insert an element at the end of the linked list.
+
+ For append method, we should consider two kinds of situations:
+
+ - if the linked list is empty, or
+ - if the linked list is not empty.
+
+ ```python
+ def append(self, data):
+ """
+ the append method will insert an element at the end of the linked list.
+ """
+ new_node = Node(data) # create a node that will be inserted
+
+ if not self.head: # the linked list is empty
+ self.head = new_node
+ return
+
+ last_node = self.head # the linked list is not empty
+ while last_node.next:
+ last_node = last_node.next
+ last_node.next = new_node
+ ```
+
+
+
+- `prepend`, the prepend method will insert an element at the beginning of the linked list.
+
+ ```python
+ def prepend(self, data):
+ """
+ it will insert an element at the beginning of the linked list.
+ """
+ new_node = Node(data)
+ new_node.next = self.head
+ self.head = new_node
+ ```
+
+
+
+- `insert_after_node`, this method will insert an element after a given node.
+
+ If we want to insert an element at somewhere(except head and tail) of the linked list, then this method will work. First, let's break down the steps of solving this kind of problem:
+
+ - First of all, we will create a new node based on the given data;
+ - Next, we need to check if the node to be inserted after is in the linked list or not(here just check whether it is a non-node);
+ - Last, we change the next pointer of the previous node to point to the new node.
+
+ ```python
+ def insert_after_node(self, pre_node, data):
+ """
+ it will insert an element after a given node.
+ """
+ if not pre_node:
+ print("Previous node is None")
+ return
+
+ new_node = Node(data)
+ new_node.next = pre_node.next
+ pre_node.next = new_node
+ ```
+
+
+
+
+#### 3). Deletion
+
+##### 3-1). Deletion by Value
+
+To delete a node in the linked list, first is to find the node to be deleted by traversing the linked list first. Then, delete that node and update the rest of pointers. However, to solve this problem, we need to handle two cases:
+
+- Node to be deleted is head, or
+- Node to be deleted is not head.
+
+
+```python
+def delete_node(self, data):
+ """
+ this method will delete the given data node if existed in the linked list
+ """
+ cur_node = self.head
+ if cur_node and cur_node.data == data: # deal with the case of head
+ self.head = cur_node.next
+ cur_node = None
+ return
+ else:
+ pre_node = None # other cases
+ while cur_node and cur_node.data != data:
+ pre_node = cur_node
+ cur_node = cur_node.next
+
+ if not cur_node: return
+ pre_node.next = cur_node.next
+ cur_node = None
+```
+
+
+
+
+##### 3-2). Deletion by position
+
+Now, let's delete a node based on position rather than value. Again, we’ll consider the above twos cases
+
+```python
+def delete_node_at_pos(self, pos):
+ """
+ it will delete a node based on given position.
+ """
+ cur_node = self.head
+ if cur_node and pos == 0: # deal with the case of head
+ self.head = cur_node.next
+ cur_node = None
+
+ count = 0 # other cases
+ pre_node = None
+ while cur_node and count != pos:
+ pre_node = cur_node
+ cur_node = cur_node.next
+ count += 1
+
+ if not cur_node: return
+ pre_node.next = cur_node.next
+ cur_node = None
+```
+
+
+
+
+#### 4). Length
+
+There are two ways to calculate the number of nodes in a linked list, which are iterative and recursive manners.
+
+##### 4-1). Iterative solution
+
+```python
+def len_iterative(self):
+ """
+ return the number of nodes in a linked list, using iterative method
+ """
+ count = 0;
+ cur_node = self.head
+ while cur_node:
+ count += 1
+ cur_node = cur_node.next
+
+ return count
+
+```
+
+##### 4-2). Recursive solution
+
+Before we start the recursive solution, let's look the following diagram that could help us understand the method more specific.
+
+
+
+For any recursive method, we need a base case. In this class, the base case is whether or not we've encountered the end of the linked list which means the `None` pointer.
+
+- if we reach the end of the linked list, it should return `0`, otherwise;
+- call the recursive method and pass in the next node and plus `1`.
+
+```python
+def len_recursive(self, node):
+ """
+ return the number of nodes in a linked list, using recursive method
+ """
+ if not node: return 0 # base case
+ return 1 + self.len_recursive(node.next)
+```
+
+
+
+
+#### 5). Node Swap
+
+Now, assume we have two keys corresponding to the data element in the nodes, we would like to swap the two nodes in a linked list, what should we do?
+
+One way to solve this is by iterating the linked list and keeping track of certain pieces of information that are going to be helpful.
+
+- We can start from the first node of the linked list and keep track of both the previous and the current node.
+- If the data element of the node that we’re on matches one of the two keys, we record the information and repeat the process for the second node
+
+There are two cases that we’ll have to cater for:
+- Node 1 and Node 2 are not head nodes.
+- Either Node 1 or Node 2 is a head node.
+
+
+```python
+def swap_nodes(self, data_1, data_2):
+ """
+ it will swap two nodes if they're existed in the linked list
+ """
+ if data_1 == data_2: return # two data are duplicate.
+
+ pre_1, cur_1 = None, self.head # looking for the data_1 and its previus node
+ while cur_1 and cur_1.data != data_1:
+ pre_1 = cur_1
+ cur_1 = cur_1.next
+
+ pre_2, cur_2 = None, self.head # looking for the data_2 and its previuos node
+ while cur_2 and cur_2.data != data_2:
+ pre_2 = cur_2
+ cur_2 = cur_2.next
+
+ if not cur_1 or not cur_2: return #cur_1 or cur_2 is none, means one of the data is no existed in linked list
+
+ if pre_1:
+ pre_1.next = cur_2
+ else: # data_1 is the head node
+ self.head = cur_2
+ if pre_2:
+ pre_2.next = cur1
+ else: # data_2 is the head node
+ self.head = cur_1
+
+ cur_2.next, cur_1.next = cur_1.next, cur_2.next # swap the two nodes.
+
+```
+
+
+
+
+#### 6). Reverse
+
+In this part, let's talk about how to reverse a singly linked list in an iterative way and a recursive way. Before getting started, let's know how to reverse a linked list first.
+
+
+
+##### 6-1). Iterative implementation
+The key idea is that we’re reversing the orientation of the arrows. For example, node A is initially pointing to node B but after we flip them, node B points to node A. The same is the case for other nodes.
+
+
+```python
+def reverse_itr(self):
+ """
+ reverse the linked list, using iterative method
+ """
+ pre, cur = None, self.head
+ while cur: # three pointers for reversing
+ nxt = cur.next;
+ cur.next = prev
+ prev = cur
+ cur = nxt
+ # new head after reversing
+ self.head = prev
+```
+
+
+
+
+##### 6-2). Recursive Implementation
+
+The crux of recursive solution is as follows:
+- the base case;
+- assume we solve the simplest problem, which in this case is to reverse just ONE pair of nodes
+- the rest nodes of the linked list will repeat above process.
+
+```python
+def reverse_rec(self):
+ """
+ it will reverse the linked list, using recursive method
+ """
+ def _reverse(pre, cur):
+ if cur:
+ _reverse(cur, cur.next)
+ cur.next = pre
+ else:
+ self.head = pre
+
+ _reverse_recursive(pre=None, cur=self.head) # setting the head, which is the pre
+```
+
+
+
+
+#### 7). Remove Duplicates
+
+We can use a hash-table or dictionary in Python to remove duplicates from a linked list.
+
+- The general idea is to go through the linked list once and keep track of all the data held at each of the nodes.
+- Use a dictionary to keep track of the data elements that we encounter to determine whether in the dictionary.
+
+
+```python
+def remove_deuplicates(self):
+ """
+ it will remove duplicate data held by nodes, using a dictionary.
+ """
+ dup_data = dict()
+ cur = self.head
+ pre = None
+
+ while cur:
+ if cur.data in dup_data: # if the data has existed in the dictionary, then removing it
+ pre.next = cur.next
+ cur = None
+ else:
+ dup_data[cur.data] = 1 # else, put the data in the dictionary
+ pre = cur
+
+ cur = pre.next
+```
+
+
+
+
+#### 8). Nth-from-last Nodes
+
+Now let's dive into the solutions:
+1. ragualer way, counting the length, break down this solution in two simple steps:
+ - Calculate the length of the linked list.
+ - Count down from the total length until n is reached.
+
+Let's look into the implementation:
+
+
+```python
+def print_nth_from_last(self, n):
+ """
+ to find the data of n-th node from the last, using length of the linked list
+ """
+ total_len = self.len_iterative()
+
+ cur = self.head
+ while cur: # countdown using total_len decrement
+ if total_len == n:
+ return cur.data
+ total -= 1
+ cur = cur.next
+ if not cur:
+ print(str(n) + "is greater than the length of the linked list.")
+ return
+```
+
+2. two pinters pattern, when fast pointer or next node of fast pointer is None, the slow pointer will reach the n
+ - `p` will point to the head node.
+ - `q` will point n nodes beyond head node.
+
+
+About two pointers pattern, the slow pointer will at the middle of the linked list, if the fast node is none or fast node's next node is none. This because, in even length fast node will be the `(length // 2 + 1)`, in odd length the fast node will be the excat middle one.
+
+Here, this pattern can be used to look for the nth-node from the last node. The idea behind this solution is that the gap between p and q is the n, when p or q.next is None
+
+
+```python
+def print_nth_from_last(self, n):
+ """
+ to find out the data of n-th node from the last, using two pointer pattern
+ """
+ p = q = self.head
+
+ count = 0
+ while q: # q points to the n-th node from the head
+ count += 1 # count += 1, because q have pointed 1 node already.
+ if count >= n: break
+ q = q.next
+
+ if not q: print(str(n) + "is greater than the length of the linked list.")
+
+ while p and q.next: #gap between p and q is n, when p or q.next is None, the n-th node from last is found
+ p = p.next
+ q = q.next
+
+ return p.data
+```
+
+Both the solutions have been made part of the `LinkedList` class. We can call the solution of the choice by passing in `1` or `2` as `method` to `print_nth_from_last(self, n, method)`.
+
diff --git a/_posts/2020-04-29-Build_a_Snake_Game_in_Python_Day_01.md b/_posts/2020-04-29-Build_a_Snake_Game_in_Python_Day_01.md
new file mode 100644
index 00000000..90afa3de
--- /dev/null
+++ b/_posts/2020-04-29-Build_a_Snake_Game_in_Python_Day_01.md
@@ -0,0 +1,73 @@
+---
+layout: default
+title: Build a snake game in Python Day-1
+date: 2020-04-29 23:30:00 -0500
+published: 2020-04-29 23:30:00 -0500
+comments: true
+tags: [Python, pygame, curses]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+## Introduction
+
+How are you guys😀! In the next following days, I'm going to introduce an interesting and small project implemented in `Python`, using some third-party libraries to create a snake game. I know it's not a big deal to complete such a project, but it really gives me a kind of enjoyment in these self-isolation days🤓.
+
+
+
+Actually, this idea is inspired by a YouTuber and he did a snake game using `curses` library which is natively supported by `Python`. You can check it out here 👉[Creating a Snake game with Python in under 5 minutes](https://www.youtube.com/watch?v=rbasThWVb-c&feature=youtu.be)👈. I followed his code and made it, below is the outcome.
+
+
+
+Honestly, I was caught by the keywords '5 minutes', it took me more than 5 minutes actually. However, I should say after completing this naive game my passion was ignited and I thought it's time for implementing a beautiful version to make me feel happy. As a result, I've just achieved this game in first version, and it will be explained in detail in the next days. Here is the glance of the game interface.
+
+
+
+I intend to introduce the development in the following steps:
+- Day1: basic ideas about `curses`
+- Day2: how to use `curses` to implement the game
+- Day3: let's analyze the game in deep using OOP ideas
+- Day4: learn pygame to draw objects and get commands
+- Day5: implement the key classes of the game
+- Day6: add background music and images to make it better
+
+Now, let's jump into the `curses` to start the journey.
+
+## About `curses`
+
+The `curses` is a library for people who want to create command-line applications. It's based on the terminal on Linux or Mac to build textual interfaces, using the keyboard to interact with users. It's a built-in `Python` library and provides lots of APIs to implement powerful functionalities. In short, there are two key concepts in this module.
+
+### 1. Initializing and closing a window
+
+We can simply think a window is a kind of canvas that contains all the objects that will display on a terminal. We can create multiple windows with different sizes and place them around your screen. If we add something on a window, we should refresh the window to display the changes and if we don't use the window anymore, we should close it properly.
+
+```python
+import curses
+
+main_scr = curses.initscr() # initialize a main screen, getting the window object
+# do something
+main_scr.refresh() # in order to update the window
+# job is done
+curses.endwin() # close the window
+```
+
+If we want to declare a window with a specific size, we should use another method called `newwin()`, this method can set the window's start coordinates, height, and width.
+
+```python
+board = curses.newwin(height, width, 0, 0) # initialize a window with the size of (height, width) on the screen (0, 0)
+```
+
+### 2. Adding and displaying characters
+
+As we know, characters can be represented either in strings or letters one by one. Text can be printed on a window by using the following statements.
+
+```python
+board.addch(0, 0, '$') # painting a character '$' on the screen (0, 0)
+board.addstr(0, 0, "Hello, snake") # putting a sentence on the screen (0, 0)
+```
+
+The above is enough for creating a simple terminal snake game, for full APIs please check official documents 👉[here](https://docs.python.org/3/library/curses.html)👈. Let's dive into more at tomorrow, see ya.
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
\ No newline at end of file
diff --git a/_posts/2020-04-30-Build_a_Snake_Game_in_Python_Day_02.md b/_posts/2020-04-30-Build_a_Snake_Game_in_Python_Day_02.md
new file mode 100644
index 00000000..bf0f08c1
--- /dev/null
+++ b/_posts/2020-04-30-Build_a_Snake_Game_in_Python_Day_02.md
@@ -0,0 +1,189 @@
+---
+layout: default
+title: Build a snake game in Python Day-2
+date: 2020-04-30 23:30:00 -0400
+published: 2020-04-30 23:30:00 -0400
+comments: true
+tags: [Python, curses]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+After knowing the building blocks of `curses`, let's jump onto our first version of the snake game🧐.
+
+
+
+Maybe you will think that it seems too abstract for you to take actions to build such a game using the previous knowledge, and the scenarios might be familar with you, if someone told you how to draw a panda in 4 steps 🤨
+
+
+
+## Analyzing the snake game
+
+Exactly right, even we know all the ways how to use tools we cannot build a house without enough preparations ahead. Let's recall the last time you played the snake game, if you have no ideas about the game, just googling it😏. Here are the major factors in this game:
+
+1. There should be three main actors in the game:
+
+- First is the main board or screen, which contains a snake and a piece of food.
+- Then is the snake, it will travel on the board and searching for food.
+- The last thing is a piece of food that can be eaten by the snake. That's all visible objects on the screen.
+
+2. Except for the above objects, we should know how the ways that users interacting with the game. In this terminal game, we assume that the user can only use four arrow keys to control the moving directions of the snake.
+
+3. Snake is controlled by users and the main purpose of the game is to eat as much as possible food, so every time a piece of food is eaten by the snake it will show on the board randomly and the snake will get lengthened due to eating the food.
+4. If the snake crashed into the boundary of the board or itself, then the game will over.
+
+As long as we have the ideas of the game, we could get started on the code, doing the coolest things😎. However, before we dive into the code, it's best to take a bird's-eye view and see everything all right:
+
+- firstly, we should initialize a screen for using as the board of the game, and I hope you have ideas about how to set a screen in `curses`;
+- then, objects, a snake and a piece of food, are drawn on the screen;
+- the last and the most complex step is the main process of the game, controlling snake to move and to eat food.
+
+## Coding time
+
+### 1. To initialize a screen
+
+First and the formost is to initialize a screen, otherwise nothing will be shown😂.
+
+```python
+import curses, random
+
+# 0. initialize a main screen of the game
+main_scr = curses.initscr()
+curses.curs_set(0) # 0.1 setting the visibility of cursor, 0-invisible, 1-normal, 2-strong
+# 1. based on the main screen to create a gaming board
+height, width = main_scr.getmaxyx() # 1.1 get the height and width of the main screen
+board = curses.newwin(height, width, 0, 0) # 1.2 create a board for gaming
+board.keypad(1) # 1.3 using keyboard for receiving commands
+board.timeout(100) # 1.4 setting for inerval, every 100 ms the board will get a char from user
+```
+
+When we have the canvas, we can paint snake and food on it right now.
+
+### 2. To paint snake and food
+
+Everything displayed on screen is tracked by using a pair of coordinate, let's say `(y, x)`. Y-asixs represents the height of the object and x-asixs represents the width of the object. If we draw a box on a screen and its coordinate is `(y, x)`, it can be explained using below figure.
+
+
+
+```python
+# 2. draw a snake on the board
+snake_y = int(height / 4) # 2.1 initial head coordinates of a snake
+snake_x = int(width / 2)
+snake = [ # 2.2 draw a snake with coordinates on the board
+ [snake_y, snake_x], # 2.3 the coordinates of head
+ [snake_y, snake_x - 1], # 2.4 the coordinates of body
+ [snake_y, snake_x - 2],
+ [snake_y, snake_x - 3],
+ [snake_y, snake_x - 4]
+]
+board.addch(snake[0][0], snake[0][1], '+') # 2.5 paint chars on the screen and update screen
+main_scr.refresh()
+
+# 3. put the first food spot on the board
+food_y = int(height / 2) # 3.1 initial position of food
+food_x = int(width / 2)
+food_pos = [food_y, food_x]
+board.addch(food_pos[0], food_pos[1], '$') # 3.2 paint the first position of food
+main_scr.refresh()
+```
+
+The next is the core of the game, which will contrl the snake to move on the board and eat the food. Before that we should set a defualt direction of the snake, then it will automatically move at first.
+
+```python
+# 4. set the default direction of the snake
+key = curses.KEY_RIGHT
+```
+
+### 3. Main process of the game
+
+#### 3.1 To Receive command from keyboard
+
+In `curses`, we use the function `window.getch() ` to get the pressed key from keyboard and it will return an integer which represents a key. [Keys are referred to by integer constants with names starting with `KEY_`](https://docs.python.org/3/library/curses.html)
+
+```python
+# 6. get command from user to control the snake
+next_key = board.getch() # 6.1 waiting for getting input from keyboard
+if next_key != -1: # 6.2 successfully received from keyboard
+ if key == curses.KEY_RIGHT and next_key != curses.KEY_LEFT \
+ or key == curses.KEY_LEFT and next_key != curses.KEY_RIGHT \
+ or key == curses.KEY_UP and next_key != curses.KEY_DOWN \
+ or key == curses.KEY_DOWN and next_key != curses.KEY_UP:
+ key = next_key
+```
+
+#### 3.2 To check the snake whehter is alive
+
+After moving the directions of the snake, we should check the status of the snake to ensure the snake is alive and the game can continue. There are two cases of ending game, one is the snake meets the boundary of the board, and the other is the snake crash itself.
+
+```python
+# 7. check whether the snake is alive
+if snake[0][0] in [0, height] or snake[0][1] in [0, width] or snake[0] in snake[1:]: # 7.1 cases for ending game
+ main_scr.keypad(0) # reset the terminal
+ curses.echo()
+ curses.nocbreak()
+ curses.endwin() # close the terminal
+ quit()
+
+# 8. draw a new head of the snake and concatenate it into the previous snake body
+snake_y = snake[0][0]
+snake_x = snake[0][1]
+if key == curses.KEY_RIGHT: # move to right
+ snake_x += 1
+elif key == curses.KEY_LEFT: # move to left
+ snake_x -= 1
+elif key == curses.KEY_UP: # move to up
+ snake_y -= 1
+elif key == curses.KEY_DOWN: # move to down
+ snake_y += 1
+
+snake.insert(0, [snake_y, snake_x]) # put the new head of snake in the list
+```
+
+#### 3.3 To check the food whether is eaten
+
+When the snake moved, the food might be eaten by the snake or not. If the snake eats the food, we should lengthen the snake and put another food on the screen, otherwise we don't change anything on the screen.
+
+```python
+# 9. check whether the food is eaten by the snake
+if snake[0] == food_pos: # 9.1 eat the food
+ food_pos = None
+ while not food_pos: # create new food randomly but excluding the coordinates in snake
+ food_y = random.randint(1, height - 1)
+ food_x = random.randint(1, width - 1)
+ food_pos = [food_y, food_x]
+ if food_pos not in snake: # avoid new coordinates in snake
+ board.addch(food_pos[0], food_pos[1], '$')
+ else:
+ food_pos = None
+else: # 9.2 not eat the food, pop the tail since we insert a new head before
+ tail = snake.pop()
+ board.addch(tail[0], tail[1], ' ')
+```
+
+Don't forget to paint changes on the screen and update it.
+
+```python
+# 10. draw the new head on screen
+board.addch(snake[0][0], snake[0][1], '+')
+main_scr.refresh()
+```
+
+#### 3.4 To animate the snake
+
+Once we achieve the functionalities of above sections in #3.1 - #3.3, we should make the snake alived on the screen, but how? The code is a little tricky and straightforward, we need to put the code in a infinite loop and boom all things have done😎.
+
+```python
+# 5. Main loop until the user lose or close the game.
+while True:
+ # code in 3.1 - 3.3
+```
+
+Finally, we achieved the first version of the game, enjoy it 🤩!
+
+
+
+See previous articles plece check 👉[here](https://cao-weiwei.github.io/posts/Build_a_Snake_Game_in_Python_Day_01/)👈. Next, we'll improve our tiny game using GUI, however, before that we should go deep in analysis using object oriented programming techniques.
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
\ No newline at end of file
diff --git a/_posts/2020-05-02-Build_a_Snake_Game_in_Python_Day_03.md b/_posts/2020-05-02-Build_a_Snake_Game_in_Python_Day_03.md
new file mode 100644
index 00000000..1bd6a2c6
--- /dev/null
+++ b/_posts/2020-05-02-Build_a_Snake_Game_in_Python_Day_03.md
@@ -0,0 +1,78 @@
+---
+layout: default
+title: Build a snake game in Python Day-3
+date: 2020-05-02 23:00:00 -0400
+published: 2020-05-02 23:00:00 -0400
+comments: true
+tags: [Python, pygame]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+Today, I'm going to introduce `pygame` in a brief way. The main idea of this post is not to explain the detials of the `pygame` but some basic ideas of gaming programming. Let's get started😃!
+
+
+
+## How objects animate
+
+Gaming programming may have similarity with making animations, the core is not to make objects move literally but the way how we updating objects coordinates.
+
+- We draw objects on the surface and updating their positions to give us a kind of illusion to think the objects is alive.
+- Another thing about to determin whether the snake is alive, to check the objects' coordinates to find whether there is overlapping, if there is intersection of different objects' coordinates, we can say they may dead since there is collision.
+
+I think above is the key concepts we should understand for further programming🤓.
+
+## How to use `pygame`
+
+Basically, there are three steps for making a game in `pygame`:
+
+- Initialize and quit a game
+- understand the coordinates in the game
+- keep game ongoing
+
+## Initialize and quit a game
+
+The first thing before making any actions is to import the `pygame` package.
+
+```python
+import pygame
+```
+
+And then we need to initialize all the `pygame` modules:
+
+```python
+pygame.init() # initialize all pygame related modules
+```
+
+After initializing all related modules, we should create a surface for showing the game. Here `pygame` providers a way to control the display window and screen.
+
+```python
+pygame.display.set_mode() # initialize a Surface to represent your drawing
+pygame.display.update() # update the screen
+```
+
+The `display.set_mode()` function creates a new Surface object that represents the actual displayed graphics. Any drawing you do to this Surface will become visible on the monitor after you call `display.update()` method which allows the screen to be updated
+
+Next is the game iteself. Once we terminate the game, modules have to be closed to clean up all resources that we used. It can be done very ease.
+
+```python
+pygame.quit() # uninstall all pygame modules, and call it before the game ends
+```
+
+We can simply think the whole process of a game as follow:
+
+
+
+## Understand the coordinates in the game
+
+The is very similar with what we learned in `curse`, please check it [here](https://cao-weiwei.github.io/posts/Build_a_Snake_Game_in_Python_Day_02/)
+
+## Keep game ongoing
+
+In order to avoid quiting the game after just starting, usually we'll add a infinite loop in the game. The basic build blocks of making object moving on the screen is as same as what we knew in `curses`library.
+
+Next, we'll dive into the detail of coding in `pygame`.
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
\ No newline at end of file
diff --git a/_posts/2020-05-03-Build_a_Snake_Game_in_Python_Day_04.md b/_posts/2020-05-03-Build_a_Snake_Game_in_Python_Day_04.md
new file mode 100644
index 00000000..7701160d
--- /dev/null
+++ b/_posts/2020-05-03-Build_a_Snake_Game_in_Python_Day_04.md
@@ -0,0 +1,76 @@
+---
+layout: default
+title: Build a snake game in Python Day-4
+date: 2020-05-03 23:00:00 -0400
+published: 2020-05-03 23:00:00 -0400
+comments: true
+tags: [Python, OOP]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+😁A snake game could contains lots of features, if we want to improve the previous work, we must have clues about how to make progress🤔. Next we will use OOP ideas to make a fancy snake game.
+
+
+
+Contrary to procedure-oriented programming where programs are designed as blocks of statements to manipulate data, object-oriented programming organizes the program to combine data and functionality and wrap it inside something called an “Object”🧐.
+
+## OOP analysis and design
+
+There is a structured approach for analyzing and designing a software product using OOP concepts. The most important thing is identifying the objects of the product and figuring out the relationships between each other. I'll explain the process of making the snake game as🥳:
+
+1. Clarifying the requirements of snake game;
+2. Drawing a diagram to show how the snake game works;
+3. Identifying the objects and defining their relationships;
+4. Making a design of the game.
+
+## Clarifying requirements
+
+🤓I'll focus on the following set of requirements while designing the snake game:
+
+- At the screen, there is a welcome interface for users to wait for a further signal, users can click either 'start' button to start a game or 'quit' button to close the window.
+- If the user clicks the 'start' button, entering a new game:
+ - Food will be randomly put on the screen
+ - A snake appears on somewhere of the screen and moving from left to right by default
+ - The user whether presses arrow keys or not, the snake will keep moving
+ - Press arrow keys to change the next direction of snake movement
+ - Each time a piece of food is eaten, the snake's body length will increase by one, the user will get one point
+ - If the food is eaten by the snake, the system will put another food randomly on the screen
+ - The snake touches the wall or itself, the game ends
+- If the user clicks the 'quit' button, the game will be terminated immediately.
+
+## Drawing process diagram and making prototype
+
+The above whole process can be drawn in a figure as:
+
+
+
+According to the process, I made mock-up of the interface.
+
+
+
+## Identifying objects and defining relationships
+
+As we knew, the most obvious objects in the game are the snake and the food. How to identify others and develop their relationships are vital points for making this game.
+
+**Board**: Everything in the game must be drawn or displayed on a board, like painting on canvas. And this board will contain a snake, food, and a score counter. It will initialize a game and end the game according to the conditions.
+
+**Snake**: A snake, in the game, actually is a list of coordinates. we have to know the start position and its length. For the snake, it can move on the screen and eat food. And sometimes, it may die due to collision with the border or itself.
+
+**Food**: As the same as the snake, it's also represented by a pair of coordinates on the screen. For the food, we should know whether it is eaten by the snake then we can draw another on the screen.
+
+**Button**: This is a kind of UI component to receive a start signal and a terminate signal from use.
+
+
+
+## Making design of the game
+
+Finally, based on previous analysis and objects we got, I made some UI of the game.
+
+
+
+Next, we'll jump onto the pygame, which is a most famous game development library in Python.
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
\ No newline at end of file
diff --git a/_posts/2020-05-05-Build_a_Snake_Game_in_Python_Day_05.md b/_posts/2020-05-05-Build_a_Snake_Game_in_Python_Day_05.md
new file mode 100644
index 00000000..7ef3c727
--- /dev/null
+++ b/_posts/2020-05-05-Build_a_Snake_Game_in_Python_Day_05.md
@@ -0,0 +1,305 @@
+---
+layout: default
+title: Build a snake game in Python Day-5
+date: 2020-05-05 23:00:00 -0400
+published: 2020-05-06 23:00:00 -0400
+comments: true
+tags: [Python, pygame]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+Now we almost have developled the entire game, just a little coding. Today, I will introduce the class implementation based on previous analysis😃!
+
+
+
+## The Snake Class
+
+🤓As we've already analyzed, the `Snake` class should do following things:
+
+- initialize a snake instance with attributes
+- get signals from keyboard to move in different directions
+- to eat the food on the screen and
+- to determine whehter the snake is alive after moving
+
+And the implementation is here:
+
+```python
+"""
+@author Weiwei Cao 2020-05-03
+This is a class of Snake.
+"""
+import Food
+import InitGame
+
+
+class Snake:
+ def __init__(self):
+ """
+ initialize a snake
+ """
+ self.default_length = 5
+ self.head_x, self.head_y = [InitGame.WIDTH // 4, InitGame.HEIGHT // 2]
+ self.snake = [[self.head_x - (i * 20), self.head_y] for i in range(self.default_length)]
+ self.current_direction = "right" # default direction of the snake
+
+ def move(self, next_direction: str) -> None:
+ """
+ receive signals from keyboard and changing coordinates of the snake
+ :param next_direction: next direction string
+ :return: None
+ """
+ # 0. change the direction of next direction is not opposite with current direct
+ if self.current_direction == "right" and next_direction != "left" \
+ or self.current_direction == "left" and next_direction != "right" \
+ or self.current_direction == "up" and next_direction != "down" \
+ or self.current_direction == "down" and next_direction != "up":
+ self.current_direction = next_direction
+
+ # 0.1 get new head position
+ new_head_x, new_head_y = self.head_x, self.head_y
+ if self.current_direction == "up":
+ new_head_x = self.head_x
+ new_head_y -= 20
+ elif self.current_direction == "down":
+ new_head_x = self.head_x
+ new_head_y += 20
+ elif self.current_direction == "left":
+ new_head_x -= 20
+ new_head_y = self.head_y
+ elif self.current_direction == "right":
+ new_head_x += 20
+ new_head_y = self.head_y
+ # 0.2 insert new head into the 1st index and update the head position
+ self.snake.insert(0, [new_head_x, new_head_y])
+ self.head_x, self.head_y = new_head_x, new_head_y
+ # 0.3 pop the last coordinate (x, y)
+ self.snake.pop()
+ print(self.snake)
+
+ def get_position(self) -> list:
+ """
+ get the coordinates of the snake = head * 1
+ :return: a list contains coordinates of a snake
+ """
+ return self.snake
+
+ def get_length(self) -> int:
+ """
+ get current length of the snake
+ :return: current length of the snake
+ """
+ return len(self.snake) - self.default_length
+
+ def eat(self, food: Food) -> None:
+ """
+ After eating a piece of food, updating the coordinates of the snake
+ :param food: an instance of Food
+ :return: None
+ """
+ food_pos = food.get_position() # get the food position
+ self.snake.insert(0, [food_pos[0], food_pos[1]]) # add food into the snake
+ self.head_x, self.head_y = food_pos[0], food_pos[1] # update the head of the snale
+
+ def is_alive(self) -> bool:
+ """
+ Check whether the snake is alive
+ :return: True if the snake is alive, otherwise False
+ """
+ # if the snake head is on the border or has collision with itself, then dead, return False
+ if self.head_x in [-InitGame.W_CELL, InitGame.WIDTH] or self.head_y in [-InitGame.H_CELL, InitGame.HEIGHT] \
+ or [self.head_x, self.head_y] in self.snake[1:]:
+ return False
+ else: # the snake is alive
+ return True
+
+```
+
+## The Food Class
+
+Basically, `Food` class will do such things:
+
+- initialize a food position
+- to determine whether current food is eaten by the food and
+- update another position for the new food
+
+```python
+"""
+@author Weiwei Cao 2020-05-03
+This is a class of Food.
+"""
+
+import random
+import InitGame
+import Snake
+
+
+class Food:
+ def __init__(self):
+ """
+ Initialize the first pair of coordinate of food
+ """
+ self.food_x = random.randint(5, InitGame.COLS - 5) * InitGame.W_CELL
+ self.food_y = random.randint(5, InitGame.ROWS - 5) * InitGame.H_CELL
+ self.food_pos = [self.food_x, self.food_y]
+
+ def update_position(self) -> None:
+ """
+ Randomly generate a new pair of coordinate for putting food
+ :return: None
+ """
+ self.food_x = random.randint(5, InitGame.COLS - 5) * InitGame.W_CELL
+ self.food_y = random.randint(5, InitGame.ROWS - 5) * InitGame.H_CELL
+ self.food_pos = [self.food_x, self.food_y]
+
+ def get_position(self) -> list:
+ """
+ Get the food position
+ :return: a list contains the food coordinate in x-axis and y-axis
+ """
+ return self.food_pos
+
+ def is_eaten(self, snake: Snake) -> bool:
+ """
+ To check the food is whether eaten by the snake
+ :param snake: an instance of Snake
+ :return: True if the food is eaten by the snake else return False
+ """
+ if self.food_pos in snake.get_position():
+ return True
+ else:
+ return False
+
+```
+
+
+
+## The Button Class
+
+There is no `Button` class in `pygame` and we need buttons to acpture mouse status and to start or quit the game, so we add following features in this class:
+
+- to set and get attributes of a button, such as the position, size and color, etc,.
+- to check the status of a button, like does the mouse is clicked in a legal area of the button
+
+```python
+"""
+@author Weiwei Cao 2020-05-03
+This is a class of Button.
+"""
+
+
+class Button:
+ def __init__(self, btn_pos: tuple, btn_size: tuple, txt_pos: tuple, txt: str):
+ """
+ Initialize a Button object with below attributes
+ :param btn_pos: a pair of coordinate, (x, y)
+ :param btn_size: a pair of size, (width, height)
+ :param txt_pos: a pair of coordinate, (x, y)
+ :param txt: a string that will displayed on button
+ """
+ self.button_pos = btn_pos
+ self.button_width = btn_size[0]
+ self.button_height = btn_size[1]
+ self.button_text_pos = txt_pos
+ self.button_txt = txt
+
+ def get_position(self) -> tuple:
+ """
+ get button start coordinate
+ :return: a tuple contains start coordinate (x, y)
+ """
+ return self.button_pos
+
+ def get_size(self) -> tuple:
+ """
+ get the size of button
+ :return: a tuple contains width and height of the button (width, height)
+ """
+ return self.button_width, self.button_height
+
+ def get_text(self) -> tuple:
+ """
+ get the text information on the button
+ :return: a tuple contains (text, [x, y])
+ """
+ return self.button_txt, self.button_text_pos
+
+ def is_hover(self, mouse_pos: tuple) -> bool:
+ """
+ To check whether the mouse is hover on the button
+ :param mouse_pos: an status of mouse object
+ :return: True if the mouse is hover on the button else return False
+ """
+ if self.button_pos[0] <= mouse_pos[0] <= (self.button_pos[0] + self.button_width) \
+ and self.button_pos[1] <= mouse_pos[1] <= (self.button_pos[1] + self.button_height):
+ return True
+ else:
+ return False
+
+ def is_click(self, click_status: tuple) -> bool:
+ """
+ To check whether the button is clicked by the mouse
+ :param click_status: status of mouse object
+ :param mouse_pos: status of mouse click
+ :return: True if the mouse is clicked on the button else return False
+ """
+ if click_status[0] == 1 or click_status[1] == 1 or click_status[2] == 1:
+ return True
+ else:
+ return False
+
+```
+
+## The core class of the game
+
+Next is the core of the game, and it contains some configurations of the game and implement key features. The framework of is should be as following:
+
+```python
+# a class initialize the game
+class Board(object):
+ def __init__(self):
+ """
+ Initialize all related modules and the screen
+ """
+ pass
+
+ def draw_objects(self, obj: object, color: tuple) -> None:
+ """
+ Draw objects on the screen
+ :param color: the color of objects
+ :param obj: instance of an object that should be drawn on the screen
+ :return: None
+ """
+ pass
+
+ def end_game(self, score: int) -> None:
+ """
+ End the current running game it will give options to user to choose a new round or exit the game
+ :param score: current game scores
+ :return: None
+ """
+ pass
+
+ def play_game(self) -> None:
+ """
+ The main part for starting a game
+ :return: None
+ """
+ pass
+
+ def intro_game(self) -> None:
+ """
+ An interface the before user starting the game.
+ If user click start, it will call the play_game() method
+ If user click quit, it will close the window
+ :return: None
+ """
+ pass
+
+```
+
+Next, we'll dive into the detail of this class .
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
\ No newline at end of file
diff --git a/_posts/2020-05-06-Build_a_Snake_Game_in_Python_Day_06.md b/_posts/2020-05-06-Build_a_Snake_Game_in_Python_Day_06.md
new file mode 100644
index 00000000..07495a56
--- /dev/null
+++ b/_posts/2020-05-06-Build_a_Snake_Game_in_Python_Day_06.md
@@ -0,0 +1,294 @@
+---
+layout: default
+title: Build a snake game in Python Day-6
+date: 2020-05-07 23:00:00 -0400
+published: 2020-05-07 23:00:00 -0400
+comments: true
+tags: [Python, pygame]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+😃Today I will introduce the vital part of the game which is the key class here.
+
+
+
+## The Board Class
+
+Before introducing the `Board` class, let's talk about some predefined variables that we will use and don't have strong connections with other class. Basically, there are the parameters about the screen size and color and texts.
+
+```python
+# Define the colors in RGB format
+BLACK = (0, 0, 0)
+WHITE = (255, 255, 255)
+BLUE = (0, 0, 255)
+BLUE_BUTTON = (80, 118, 249)
+GREEN = (80, 255, 120)
+GREEN_BG = (142, 204, 57)
+BRI_GREEN = (200, 255, 200)
+DARK_GREEN = (50, 150, 50)
+RED = (200, 0, 0)
+RED_BUTTON = (255, 50, 50)
+BRI_RED = (255, 200, 200)
+DARK_RED = (150, 50, 50)
+BLOODY = (100, 0, 0)
+GREY = (100, 100, 100)
+
+# Define the size of the window and cell
+WIDTH, HEIGHT = 800, 600
+ROWS, COLS = 30, 40
+W_CELL, H_CELL = WIDTH // COLS, HEIGHT // ROWS
+
+# Background songs and images
+BGM = "bgm_1.wav"
+BG_PIC = "page_1.png"
+
+# Speed
+EASY = 10
+MEDIUM = 20
+HARD = 50
+
+# Captions
+WELCOME_TXT = "Welcome to Play Snake!"
+GAME_OVER_TXT = "Game Over!"
+START_BUTTON_TXT = "Start"
+QUIT_BUTTON_TXT = "Quit"
+
+# Fonts
+ARIAL = "arial"
+GEORGIA = "georgiaboldttf"
+```
+
+The next is the main part here:
+
+1. first is the construtor of `Board` class, we define the game and other essential variables here.
+
+```python
+ def __init__(self):
+ """
+ Initialize all related modules and the screen
+ """
+ # 0. initialize all modules and music
+ pygame.init()
+ pygame.mixer.init()
+ # 1. set a screen
+ self.window_size = WIDTH, HEIGHT
+ self.window = pygame.display.set_mode(self.window_size)
+ pygame.display.set_caption("Snake v1.0")
+ self.window.fill(GREY)
+ self.clock = pygame.time.Clock()
+ # 2. set game status
+ self.is_quit = False
+ self.is_start = False
+ # 3. set font and image
+ self.title_font = pygame.font.SysFont(GEORGIA, 55)
+ self.button_font = pygame.font.SysFont(GEORGIA, 20)
+ self.normal_font = pygame.font.SysFont(GEORGIA, 30)
+ self.bg_pic = pygame.image.load(BG_PIC)
+ self.btn_start = Button.Button((220, 450), (110, 40), (250, 457), START_BUTTON_TXT)
+ self.btn_quit = Button.Button((520, 450), (110, 40), (550, 457), QUIT_BUTTON_TXT)
+```
+
+2. The next is a method that draws objects on the board. There are three types of objects, `Food`, `Snake` and `Button`. So, this method can draw any instances of the given classes.
+
+```python
+ def draw_objects(self, obj: object, color: tuple) -> None:
+ """
+ Draw objects on the screen
+ :param color: the color of objects
+ :param obj: instance of an object that should be drawn on the screen
+ :return: None
+ """
+ if isinstance(obj, Food.Food): # case 1: draw food
+ food_pos = obj.get_position()
+ pygame.draw.rect(self.window, color, (food_pos[0], food_pos[1], W_CELL, H_CELL))
+ elif isinstance(obj, Snake.Snake): # case 2: draw a snake
+ snake_pos = obj.get_position()
+ for left, top in snake_pos:
+ pygame.draw.rect(self.window, color, (left, top, W_CELL, H_CELL), 2) # rect=(left,top,w_cell,h_cell), border of snake
+ elif isinstance(obj, Button.Button): # case 3: draw a button
+ button_pos = obj.get_position()
+ button_size = obj.get_size()
+ button_txt = obj.get_text()
+ pygame.draw.rect(self.window, color, (button_pos[0], button_pos[1], button_size[0], button_size[1])) # button
+ pygame.draw.rect(self.window, WHITE, (button_pos[0], button_pos[1], button_size[0], button_size[1]), 2) # border
+ text_font = self.button_font.render(button_txt[0], True, WHITE) # text
+ self.window.blit(text_font, button_txt[1])
+```
+
+3. This `intro_game`method will do the jobs at very beginning, the interface you see a snake and an apple with two buttons. If user click the start button, it will change the game sataus and call the `play_game`method, other will it will directly close the window to quit the game.
+
+```python
+ def intro_game(self) -> None:
+ """
+ An interface the before user starting the game.
+ If user click start, it will call the play_game() method
+ If user click quit, it will close the window
+ :return: None
+ """
+ # 0. to check which buttons are clicked
+ while not self.is_start:
+ # 0.1. draw welcome screen
+ self.clock.tick(HARD)
+ text = self.title_font.render(WELCOME_TXT, True, WHITE)
+ self.window.blit(self.bg_pic, (0, 0))
+ self.window.blit(text, (60, 110))
+ # 0.2 listen to the events
+ for event in pygame.event.get():# set background font and put it on the screen
+ print(event)
+ if event.type == pygame.QUIT:
+ pygame.quit()
+ quit()
+
+ # draw buttons including 'start' and 'quit' and listening mouse actions
+ mouse_pos = pygame.mouse.get_pos()
+ mouse_click = pygame.mouse.get_pressed()
+ self.draw_objects(self.btn_start, BLUE_BUTTON)
+ self.draw_objects(self.btn_quit, RED_BUTTON)
+ if self.btn_start.is_hover(mouse_pos):
+ if self.btn_start.is_click(mouse_click):
+ self.is_start = True
+ break
+ else:
+ self.draw_objects(self.btn_start, BLUE)
+ if self.btn_quit.is_hover(mouse_pos):
+ if self.btn_quit.is_click(mouse_click):
+ self.is_quit = True
+ break
+ else:
+ self.draw_objects(self.btn_quit, RED)
+ # update the screen
+ pygame.display.update()
+
+ if self.is_start:
+ self.play_game()
+ self.is_start = False
+ if self.is_quit: pygame.quit()
+```
+
+4. When the start button is clicked, the it will show a moving snake and food on the screen. User can use arrow keys to move the snake for eating food. This process is finished by following piece of code:
+
+```python
+ def play_game(self) -> None:
+ """
+ The main part for starting a game
+ :return: None
+ """
+ # 0. play background music
+ bgm_obj = pygame.mixer.Sound(BGM)
+ bgm_obj.play(-1)
+ # 1. get food and snake coordinates and draw objects on the screen
+ food = Food.Food()
+ snake = Snake.Snake()
+ self.draw_objects(food, RED_BUTTON)
+ self.draw_objects(snake, BLUE_BUTTON)
+ # 2. others settings
+ direction = "right" # default next direction of the snake
+ score = snake.get_length() # get current length of the snake
+ clock = pygame.time.Clock() # generate a Clock Object for setting the speed of snake
+ # 3. Main loop until the user lose or close the game.
+ while not self.is_quit:
+ # 3.0 This limits the while loop to a max of HARD times per second.
+ clock.tick(HARD)
+ # 3.1 Draw current scores on the left top of the screen
+ self.window.fill(GREEN_BG)
+ score = snake.get_length()
+ total_score = self.button_font.render("Scores: " + str(score), True, WHITE)
+ self.window.blit(total_score, (5, 0))
+ # 3.2 Listen to the event,
+ # to determine the game whether is still working or getting keys from users
+ for event in pygame.event.get():
+ if event.type == pygame.QUIT: # close the game
+ print(f"event.type = {event.type}")
+ self.is_quit = True
+ break
+
+ if event.type == pygame.KEYDOWN: # receive command
+ if event.key == pygame.K_RIGHT:
+ print("*" * 30 + "right" + "*" * 30)
+ direction = "right"
+ elif event.key == pygame.K_LEFT:
+ print("*" * 30 + "left" + "*" * 30)
+ direction = "left"
+ elif event.key == pygame.K_UP:
+ print("*" * 30 + "up" + "*" * 30)
+ direction = "up"
+ elif event.key == pygame.K_DOWN:
+ print("*" * 30 + "down" + "*" * 30)
+ direction = "down"
+ # 3.3 update coordinates of the snake after receiving command from user
+ print('-' * 70)
+ snake.move(direction)
+ if not snake.is_alive(): # check whether the snake is sill alive
+ break
+ if food.is_eaten(snake): # check whether the food is still exists
+ snake.eat(food)
+ food.update_position()
+ while food.is_eaten(snake): food.update_position() # put another food except the range of snake
+ # 3.4 all is good then draw the snake and food
+ self.draw_objects(snake, BLUE_BUTTON)
+ self.draw_objects(food, RED_BUTTON)
+ # 3.5 update the screen with what we've drawn
+ pygame.display.update()
+
+ # 4. close the game
+ if self.is_quit:
+ pygame.quit()
+ else:
+ bgm_obj.stop()
+ self.end_game(score)
+```
+
+5. If the snake doesn't alive anymore the current round is over, user can play again or quit the game, this job is done by the method of `end_game`:
+
+```python
+ def end_game(self, score: int) -> None:
+ """
+ End the current running game it will give options to user to choose a new round or exit the game
+ :param score: current game scores
+ :return: None
+ """
+ # 0. set a game over screen, including text and its color position
+ self.window.fill(GREEN_BG)
+ text = self.title_font.render(GAME_OVER_TXT, True, WHITE)
+ self.window.blit(text, (240, 200))
+ total_score = self.normal_font.render("Total Scores: " + str(score), True, WHITE)
+ self.window.blit(total_score, (290, 290))
+ pygame.display.update()
+
+ while not self.is_quit:
+ for event in pygame.event.get(): # wait for clicking to close the window
+ if event.type == pygame.QUIT:
+ self.is_quit = True
+ break
+ # draw buttons including 'start' and 'quit' and listening mouse actions
+ mouse_pos = pygame.mouse.get_pos() # get current moust status
+ mouse_click = pygame.mouse.get_pressed()
+ self.draw_objects(self.btn_start, BLUE_BUTTON) # draw buttons
+ self.draw_objects(self.btn_quit, RED_BUTTON)
+ if self.btn_start.is_hover(mouse_pos): # if click 'start'
+ if self.btn_start.is_click(mouse_click):
+ self.is_start = True
+ break
+ else:
+ self.draw_objects(self.btn_start, BLUE)
+ if self.btn_quit.is_hover(mouse_pos): # if click 'quit
+ if self.btn_quit.is_click(mouse_click):
+ self.is_quit = True
+ break
+ else:
+ self.draw_objects(self.btn_quit, RED)
+ # update the screen
+ pygame.display.update() # update objects on the screen
+
+ if self.is_start: self.play_game()
+ if self.is_quit: pygame.quit()
+```
+
+Above is all the code of snake game, hope it helpful for you 🥳
+
+
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
\ No newline at end of file
diff --git a/_posts/2020-05-09-Heap.md b/_posts/2020-05-09-Heap.md
new file mode 100644
index 00000000..6d9ac04b
--- /dev/null
+++ b/_posts/2020-05-09-Heap.md
@@ -0,0 +1,79 @@
+---
+layout: default
+title: Data Structure - Quick Review of Priority Queue
+date: 2020-05-09 23:00:00 -0400
+published: 2020-05-09 23:00:00 -0400
+comments: true
+tags: [Heap, Data Structure and Algorithm]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+## Priority Queue
+
+A **priority queue** is an abstract data type, which has the behaviour somewhat like a queue. However, the first-out element is not the first-in element on time serial but the element with the highest priority in the queue. You can think it is a container holds priorities.
+
+
+
+As the priority queue is an ADT, it can be implemented in many of the data structures, such as arrays, linked lists and so on. However, the most efficient ways to achieve such data type is using **heap**. Let's move to the next parts to see the reasons.
+
+## Heap
+
+A **heap** is a binary tree with the following conditions:
+
+- a complete binary tree, and
+- for each node in the tree, the value of parent is greater (less) than or equal to the values in its children.
+
+Typically, **heap** is implemented using array and the index starting at position 1. The first element in a heap it the root which is `heap[1]`.
+
+Since it's a complete binary tree, for a node at index `i`, its left child should be at index `2*i` and the right child is at `2*i+1`.
+
+The height of heap will be `logN`.
+
+### Inserting a node in heap
+
+For inserting a node into a heap, we always start the operation at the last index since we want to keep the character of making it is a complete binary tree. The process as following:
+
+- Add a new element always at the end of the index, and then
+ - Compare the value of new added element with its parent (at index / 2)
+ - if the new greater than its parent, then swap them;
+- repeat this until it hite the root of the heap
+
+Since the height of a heap is `logN`, this operations's time complexity is `O(logN)`.
+
+
+
+### Deleting a node from heap
+
+Always delete the first element in a heap and move the last into the first index to keep the skeleton.
+
+- for current root node, pick the largest children and comparing with it
+ - if the root greater than its largest children, then STOP which means it's a valid heap;
+ - else swap them and
+
+- repeat this until it hite the end of the heap
+
+
+
+Since the height of a heap is `logN`, this operations's time complexity is `O(logN)`.
+
+### Creating a heap
+
+There are two ways to generate a Heap, one native way is using previous insertion method. we repeat inserting for `N` times and it will form a Heap as a result. However, this *Top-down* fashion costs `O (N * logN)` time which is not efficient.
+
+Another method is call `heapify`, which using *Bottom-up*, takes linear time `O(N)`. The idea is to generate the heap in-place from the backward determining each subtree whehter form a valid subheap . The process as following:
+
+
+
+- first, obviously the last row of leaves must be legal heap for itself.
+- Then move up, for each subtree, we use the properties to adjust the nodes to make them form a valid subheap.
+
+### Heap sort
+
+Heapsort uses a heap to sort the objects. As we know, each time for deletion in a heap, we can get the max/min element, then for a heap contains `N` elements, we perform deletion for `N` times will get a sorted output. As deletion method take `O(logN)` time, heap sort runs in `O(N * logN)`.
+
+
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
diff --git a/_posts/2020-05-15-Search Trie.md b/_posts/2020-05-15-Search Trie.md
new file mode 100644
index 00000000..1b46b43c
--- /dev/null
+++ b/_posts/2020-05-15-Search Trie.md
@@ -0,0 +1,190 @@
+---
+layout: default
+title: Data Structure - Trie in Python
+published: 2020-05-15 23:00:00 -0400
+comments: true
+tags: [Search, Trie, String, Data Structure and Algorithm, Python]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+
+
+Today, let’s check a data structure named `Trie`. It’s also called search trie or prefix trie, which is mainly used to handle string related problems.
+
+
+
+As we have known, the balanced binary search tree will give us `O(logN)` time complexity in search/insert/delete, which is efficient. We can use this idea to create a variant of the tree, called `Trie` to improve the efficiency of queries in strings, like searching prefix of a word. `Trie` is very useful in our daily usage of computers like auto-complete in search engines.
+
+
+
+For example, for words "apple", "ape", "apply", we will pre-process all the characters and save each alphabet in a trie node to form a trie as below:
+
+
+
+As we can see, even "apple" and "apply" are two words, we only store prefix "appl" once. The ame prefix of a string will show only once in a `Trie` and this property will help us:
+
+- to find common prefix by a given string and
+- to get all suffix by a given string efficiently.
+
+For example, suppose we have a `Trie` build by all alphabets, which means the size of each level is `26`. If we want to find a word "book", we just need to search `26` * `4` times. To be more mathematicly, we can say search in a `Trie` will cost `O(K * M)`, `K` is the size of the alphabet and `M` is the length of given string parameter of the operation.
+
+Next, let's jump onto how to implement this in `Python`.
+
+## Trie Node
+
+Before we design the `Trie`, it's better to give the data structure of trie node. A trie node is to store the data, which holds any additional data associated with the current node and a flag to represent that a word ends at this node.
+
+```python
+class TrieNode:
+ def __init__(self):
+ """
+ Initialize a trie node
+ """
+ self.data = dict()
+ self.is_end = False
+```
+
+Now let's look at the previous `Trie` in detail, below is what we will create in Python:
+
+
+
+## Trie
+
+During the class `Trie`, we need a root to start operations.
+
+```python
+class Trie:
+ def __init__(self):
+ """
+ Initialize a root of a trie
+ """
+ self.root = TrieNode()
+```
+
+
+
+### Insertion
+
+The `insert()` method will put a word in the `Tire` and set the flag as `True` which means current path is representing a complete word.
+
+```python
+ def insert(self, word: str) -> None:
+ """
+ Insert a word into a trie
+ :param word: str
+ :return: None
+ """
+ cur_node = self.root
+ for w in word:
+ if w not in cur_node.data.keys():
+ cur_node.data[w] = TrieNode() # if a character is new to the trie, just append it as a new trie node
+ cur_node = cur_node.data.get(w) # else going deeper until at the end of a path
+ cur_node.is_end = True
+```
+
+### Search
+
+`search()` method will check a word whether in the `Tries`, if existed return `True`, otherwise return `False`. It will go through the dictionary by checking the keys.
+
+```python
+ def search(self, word: str) -> bool:
+ """
+ Search the word whether exists in the trie
+ :param word: str
+ :return: True if the word exists otherwise False
+ """
+ cur_node = self.root
+ for w in word:
+ if w not in cur_node.data.keys():
+ return False # if cur_node doesn't contain this character, which means given word doesn't exist in trie
+ cur_node = cur_node.data.get(w) # going deeper for search the character
+ return cur_node.is_end
+```
+
+
+
+### Check prefix
+
+This method will identify whether the given prefix exist in a `Trie`.
+
+```python
+ def start_with(self, prefix: str) -> bool:
+ """
+ To determine whether there is any word in the trie that starts with the given prefix.
+ :param prefix: str
+ :return: True if the prefix exists otherwise False
+ """
+ cur_node = self.root
+ for w in prefix:
+ if w not in cur_node.data.keys():
+ return False
+ cur_node = cur_node.data.get(w)
+ return True
+```
+
+
+
+### Auto complete
+
+This is not a strict feature of auto-complete, however it basically implements the key feature which will return a list contains number of words of the given prefix.
+
+The logic is:
+
+- do some edge cases check first
+- then using backtracking techniques to enumerate all possible words
+
+```python
+ def auto_complete(self, prefix: str) -> list:
+ """
+ get all words started with given prefix
+ :param prefix: str
+ :return: a list of words in the trie that starts with given prefix
+ """
+
+ def _get_keys(word: str, node: TrieNode) -> list:
+ """
+ using backtracking technique to collect all words with given prefix
+ :param word: str
+ :param node: TrieNode
+ :return: a list of words in the trie that starts with given prefix
+ """
+ word_list = []
+ if node.is_end: # at the end of the search trie, put a word into the answer list
+ word_list.append(word)
+ for key in node.data.keys(): # go back to traversal all possible paths
+ word_list.extend(_get_keys(word + key, node.data.get(key)))
+ return word_list
+
+ words = []
+ if not self.start_with(prefix):
+ # no such prefix in the trie, return empty list
+ return words
+ elif self.search(prefix):
+ # or the prefix is a complete word, just return a list contains the prefix
+ words.append(prefix)
+ return words
+ else:
+ # to search all the words in the trie with given prefix
+ cur_node = self.root
+ for chars in prefix: # to find the last ancestor in the trie
+ cur_node = cur_node.data.get(chars)
+ return _get_keys(prefix, cur_node)
+
+```
+
+
+
+Here are some references may help you 🤓:
+
+[Trie | Insert and Search](https://www.geeksforgeeks.org/trie-insert-and-search)
+
+[Trie](https://www.interviewcake.com/concept/java/trie)
+
+If you want to assess your understanding of `Trie`, please check this👉[208. Implement Trie (Prefix Tree)](https://leetcode.com/problems/implement-trie-prefix-tree/)
+
+
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
diff --git a/_posts/2020-06-12-Brute_Force_Searching.md b/_posts/2020-06-12-Brute_Force_Searching.md
new file mode 100644
index 00000000..1e15fe42
--- /dev/null
+++ b/_posts/2020-06-12-Brute_Force_Searching.md
@@ -0,0 +1,103 @@
+---
+layout: default
+title: Algorithms - Brute Force Searching
+published: 2020-06-12 13:00:00 -0400
+comments: true
+tags: [Search, Brute Force, Backtracking, DFS, Branch and Bound, BFS, Data Structure and Algorithm]
+github: "https://github.com/cao-weiwei/"
+noimage: true
+---
+
+
+
+`Brute Force Searching` strategy is a useful problem-solving approach that *systematically enumerates* *all possible candidates* for the solution and then selecting the proper candidates as the result set.
+
+## Introduction
+
+It is usually used to *search* all possible results for a specific problem, and if there are *overlapping* subproblems, `Greedy` or `Dynamic Programming` skills can be applied based on the brute force process to get optimal solutions.
+
+`Backtracking` and `Branch and Bound` are two strategies for applying the Brute Force Searching approach to find all the solutions. Those two methods are also as known as *`DFS`* and *`BFS`* strategy separately.
+
+Before we dive into more details of them, let’s familiar with two useful techniques for utilizing those strategies. The first thing is called `State Space Tree`, which can represent all possible solutions using a tree structure in which successive states of an instance are considered. The second is `Bounding Function`, it is nothing but conditions to terminate invalid nodes in the tree so that the desired candidates are selected for goals and killing unnecessary branches of the tree to save time.
+
+## Backtracking
+
+The logic of `Backtracking` is that searching from the root of a state space tree to get all the promising leaves which are the results meeting the constrains.
+
+1. Firstly, take one element from the given pool, and
+
+2. Then, checking the bounding function (conditions) to see if current exploration can go further.
+
+3. If OK, then take one of the remaining elements from the pool to generate an intermediate result, and then repeat #2, until reaching the end of the tree.
+
+4. After traversing this leaf, we need to go back to the previous step, so we should remove this point added to the result.
+
+Below is an example of applying backtracking method to get all possible results.
+
+
+
+As the above figure showing, the state space tree displays how `Backtracking` method working and obvious it’s a type of `DFS` strategy and using bounding function to prune the tree.
+
+- `DFS`: `pre-order` occurs when making choice from pool to select an element, and `post-order` occurs when revoking the choice.
+
+- Pruning state space tree to avoid useless searching before `pre-order` operation and adding current path into the result set.
+
+The `Backtracking` can be broken down into following two parts in pseudocode
+
+```python
+def backtrack(current_path, depth, pool):
+ # 0. base case
+ if depth reaches the bottom:
+ if current_path is a promising path:
+ solutions.append(current_path)
+ return
+ # 1. visiting all nodes in state space tree
+ for element in pool:
+ if element is not a valid choice: continue
+ else:
+ current_path.append(element) # make choice
+ backtrack(current_path, depth + 1, pool) # go to next depth
+ current_path.pop(element) # move back, cancelling the choice
+```
+
+The difference between traditional `DFS` and the above pseudocode is that how to avoid visiting elements that have been searched. Actually, the way how an element is selected from the pool and the validation of the currently selected element can complete such a thing. Here are some exercises for this pattern🤓
+
+- Permutation
+
+[46. Permutations](https://leetcode.com/problems/permutations/)
+
+[47. Permutations II](https://leetcode.com/problems/permutations-ii/)
+
+[39. Combination Sum](https://leetcode.com/problems/combination-sum/)
+
+[40. Combination Sum II](https://leetcode.com/problems/combination-sum-ii/)
+
+- Combination
+
+[78. Subsets](https://leetcode.com/problems/subsets/)
+
+[90. Subsets II](https://leetcode.com/problems/subsets-ii/)
+
+- DFS
+
+[494. Target Sum](https://leetcode.com/problems/target-sum/)
+
+## Branch and bound
+
+Another strategy using brute force searching is `Branch and Bound`, which follows `BFS` pattern to find [all possible] candidates of the solution. The logic is similar with `BFS`, we start from the root of state space tree and put that element into a queue to generate next level nodes and exploring breadthwise. The following figure shows an instance that how `BFS` works to get the answer.
+
+
+
+This method is often used to find a possible shortest path, and what modifications are needed is the bounding functions. If you want to assess your understanding, please check this👉:
+
+[752. Open the Lock](https://leetcode.com/problems/open-the-lock/)
+
+[279. Perfect Squares](https://leetcode.com/problems/perfect-squares/)
+
+[131. Palindrome Partitioning](https://leetcode.com/problems/palindrome-partitioning/)
+
+
+
+---
+
+If you like my articles please give me a star or leave comments below, thanks!
diff --git a/assets/.DS_Store b/assets/.DS_Store
new file mode 100644
index 00000000..3fd40096
Binary files /dev/null and b/assets/.DS_Store differ
diff --git a/assets/css/.DS_Store b/assets/css/.DS_Store
new file mode 100644
index 00000000..c7287258
Binary files /dev/null and b/assets/css/.DS_Store differ
diff --git a/assets/css/font-awesome/.DS_Store b/assets/css/font-awesome/.DS_Store
new file mode 100644
index 00000000..3489a9e0
Binary files /dev/null and b/assets/css/font-awesome/.DS_Store differ
diff --git a/assets/css/main.css b/assets/css/main.css
index b5cfb7e3..35c07818 100644
--- a/assets/css/main.css
+++ b/assets/css/main.css
@@ -11,7 +11,7 @@ body {
}
-h1 {
+/* h1 {
counter-reset: h2counter;
}
@@ -27,14 +27,15 @@ h2:before {
h3:before {
content: counter(h2counter) "." counter(h3counter) ".\0000a0\0000a0";
counter-increment: h3counter;
-}
+} */
.container {
font-size: 1.3em;
}
.jumbotron {
- background: #719d7d;
+ /* background: #719d7d; */
+ background: #2196f3;
}
.jumbotron h1 {
@@ -175,4 +176,4 @@ h3:before {
color: #ffffff;
background-color: #12421f;
}
-}
\ No newline at end of file
+}
diff --git a/assets/images/-top.jpg b/assets/images/-top.jpg
new file mode 100644
index 00000000..ed3527b8
Binary files /dev/null and b/assets/images/-top.jpg differ
diff --git a/assets/images/.DS_Store b/assets/images/.DS_Store
new file mode 100644
index 00000000..d5e9eb0b
Binary files /dev/null and b/assets/images/.DS_Store differ
diff --git a/assets/images/posts/.DS_Store b/assets/images/posts/.DS_Store
new file mode 100644
index 00000000..905b0ad7
Binary files /dev/null and b/assets/images/posts/.DS_Store differ
diff --git a/assets/images/posts/Brute_Force_Searching/01_backtracking-state_space_tree__1_.png b/assets/images/posts/Brute_Force_Searching/01_backtracking-state_space_tree__1_.png
new file mode 100644
index 00000000..a83fa3b6
Binary files /dev/null and b/assets/images/posts/Brute_Force_Searching/01_backtracking-state_space_tree__1_.png differ
diff --git a/assets/images/posts/Brute_Force_Searching/02_branch_and_bound.png b/assets/images/posts/Brute_Force_Searching/02_branch_and_bound.png
new file mode 100644
index 00000000..fea4247e
Binary files /dev/null and b/assets/images/posts/Brute_Force_Searching/02_branch_and_bound.png differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/01_how_to_drawa_panda_in_4_steps.png b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/01_how_to_drawa_panda_in_4_steps.png
new file mode 100644
index 00000000..29eae8fe
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/01_how_to_drawa_panda_in_4_steps.png differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/snake_01_curse.gif b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/snake_01_curse.gif
new file mode 100644
index 00000000..c5c12810
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/snake_01_curse.gif differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/snake_02_1st_glance.gif b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/snake_02_1st_glance.gif
new file mode 100644
index 00000000..f89d9419
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_01/snake_02_1st_glance.gif differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/.DS_Store b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/.DS_Store
new file mode 100644
index 00000000..5008ddfc
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/.DS_Store differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/01_how_to_drawa_panda_in_4_steps.png b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/01_how_to_drawa_panda_in_4_steps.png
new file mode 100644
index 00000000..c0e97469
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/01_how_to_drawa_panda_in_4_steps.png differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/02_coordinate.png b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/02_coordinate.png
new file mode 100644
index 00000000..23121a20
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_02/02_coordinate.png differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/01_snake_game_flow.png b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/01_snake_game_flow.png
new file mode 100644
index 00000000..14d38da4
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/01_snake_game_flow.png differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/02_snake_game_prototype.png b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/02_snake_game_prototype.png
new file mode 100644
index 00000000..24f65835
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/02_snake_game_prototype.png differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/03_class_diagram.png b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/03_class_diagram.png
new file mode 100644
index 00000000..cbeab508
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/03_class_diagram.png differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/04_demo-ui.png b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/04_demo-ui.png
new file mode 100644
index 00000000..681ea4c5
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_03/04_demo-ui.png differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_04/.DS_Store b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_04/.DS_Store
new file mode 100644
index 00000000..5008ddfc
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_04/.DS_Store differ
diff --git a/assets/images/posts/Build_a_Snake_Game_in_Python_Day_04/01_pygame_flow.png b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_04/01_pygame_flow.png
new file mode 100644
index 00000000..203955c9
Binary files /dev/null and b/assets/images/posts/Build_a_Snake_Game_in_Python_Day_04/01_pygame_flow.png differ
diff --git a/assets/images/posts/Heap/.DS_Store b/assets/images/posts/Heap/.DS_Store
new file mode 100644
index 00000000..5008ddfc
Binary files /dev/null and b/assets/images/posts/Heap/.DS_Store differ
diff --git a/assets/images/posts/Heap/01_insert_node_into_heap.png b/assets/images/posts/Heap/01_insert_node_into_heap.png
new file mode 100644
index 00000000..f424329e
Binary files /dev/null and b/assets/images/posts/Heap/01_insert_node_into_heap.png differ
diff --git a/assets/images/posts/Heap/02_delete_node_from_heap.png b/assets/images/posts/Heap/02_delete_node_from_heap.png
new file mode 100644
index 00000000..1f793416
Binary files /dev/null and b/assets/images/posts/Heap/02_delete_node_from_heap.png differ
diff --git a/assets/images/posts/Heap/03_create_a_heap.png b/assets/images/posts/Heap/03_create_a_heap.png
new file mode 100644
index 00000000..a89da240
Binary files /dev/null and b/assets/images/posts/Heap/03_create_a_heap.png differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2761-\346\234\200\347\273\210\347\273\223\346\236\234\347\244\272\344\276\213.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2761-\346\234\200\347\273\210\347\273\223\346\236\234\347\244\272\344\276\213.png"
new file mode 100644
index 00000000..bc75ba02
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2761-\346\234\200\347\273\210\347\273\223\346\236\234\347\244\272\344\276\213.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27610-\350\275\246\346\254\241\344\277\241\346\201\257js\346\226\207\344\273\266.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27610-\350\275\246\346\254\241\344\277\241\346\201\257js\346\226\207\344\273\266.png"
new file mode 100644
index 00000000..3dc31fd3
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27610-\350\275\246\346\254\241\344\277\241\346\201\257js\346\226\207\344\273\266.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27611-\345\235\220\345\270\255\347\261\273\345\210\253\344\277\241\346\201\257\344\273\243\347\240\201.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27611-\345\235\220\345\270\255\347\261\273\345\210\253\344\277\241\346\201\257\344\273\243\347\240\201.png"
new file mode 100644
index 00000000..b2710f52
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27611-\345\235\220\345\270\255\347\261\273\345\210\253\344\277\241\346\201\257\344\273\243\347\240\201.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27612-\350\275\246\346\254\241\344\277\241\346\201\257\344\273\243\347\240\201.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27612-\350\275\246\346\254\241\344\277\241\346\201\257\344\273\243\347\240\201.png"
new file mode 100644
index 00000000..b14bd8f3
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27612-\350\275\246\346\254\241\344\277\241\346\201\257\344\273\243\347\240\201.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27613-\350\275\246\346\254\241\344\277\241\346\201\257\347\273\223\346\236\234\345\261\225\347\244\272.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27613-\350\275\246\346\254\241\344\277\241\346\201\257\347\273\223\346\236\234\345\261\225\347\244\272.png"
new file mode 100644
index 00000000..c53dd083
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27613-\350\275\246\346\254\241\344\277\241\346\201\257\347\273\223\346\236\234\345\261\225\347\244\272.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27614-\350\275\246\346\254\241\344\277\241\346\201\257\347\273\223\346\236\234\345\261\225\347\244\272-12306\347\275\221\347\253\231.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27614-\350\275\246\346\254\241\344\277\241\346\201\257\347\273\223\346\236\234\345\261\225\347\244\272-12306\347\275\221\347\253\231.png"
new file mode 100644
index 00000000..d61d469c
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\27614-\350\275\246\346\254\241\344\277\241\346\201\257\347\273\223\346\236\234\345\261\225\347\244\272-12306\347\275\221\347\253\231.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2762-docopt\347\244\272\344\276\213.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2762-docopt\347\244\272\344\276\213.png"
new file mode 100644
index 00000000..a93be6af
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2762-docopt\347\244\272\344\276\213.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2763-\346\237\245\347\234\213\345\210\227\350\275\246\344\277\241\346\201\257\346\216\245\345\217\243.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2763-\346\237\245\347\234\213\345\210\227\350\275\246\344\277\241\346\201\257\346\216\245\345\217\243.png"
new file mode 100644
index 00000000..0bf1b945
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2763-\346\237\245\347\234\213\345\210\227\350\275\246\344\277\241\346\201\257\346\216\245\345\217\243.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2764-\345\210\227\350\275\246\344\277\241\346\201\257\346\216\245\345\217\243.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2764-\345\210\227\350\275\246\344\277\241\346\201\257\346\216\245\345\217\243.png"
new file mode 100644
index 00000000..779ea84d
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2764-\345\210\227\350\275\246\344\277\241\346\201\257\346\216\245\345\217\243.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2765-\345\223\215\345\272\224\347\232\204\345\210\227\350\275\246\344\277\241\346\201\257.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2765-\345\223\215\345\272\224\347\232\204\345\210\227\350\275\246\344\277\241\346\201\257.png"
new file mode 100644
index 00000000..22753633
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2765-\345\223\215\345\272\224\347\232\204\345\210\227\350\275\246\344\277\241\346\201\257.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2766-\350\275\246\347\253\231\344\270\255\350\213\261\346\226\207\345\257\271\347\205\247\346\226\207\344\273\266.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2766-\350\275\246\347\253\231\344\270\255\350\213\261\346\226\207\345\257\271\347\205\247\346\226\207\344\273\266.png"
new file mode 100644
index 00000000..425ef005
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2766-\350\275\246\347\253\231\344\270\255\350\213\261\346\226\207\345\257\271\347\205\247\346\226\207\344\273\266.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2767-stations\345\210\235\346\255\245\345\244\204\347\220\206.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2767-stations\345\210\235\346\255\245\345\244\204\347\220\206.png"
new file mode 100644
index 00000000..7cd17cda
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2767-stations\345\210\235\346\255\245\345\244\204\347\220\206.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2768-stations\345\255\227\345\205\270.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2768-stations\345\255\227\345\205\270.png"
new file mode 100644
index 00000000..c5802183
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2768-stations\345\255\227\345\205\270.png" differ
diff --git "a/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2769-\350\275\246\346\254\241\344\277\241\346\201\257\350\277\224\345\233\236\347\273\223\346\236\234.png" "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2769-\350\275\246\346\254\241\344\277\241\346\201\257\350\277\224\345\233\236\347\273\223\346\236\234.png"
new file mode 100644
index 00000000..eafb2a57
Binary files /dev/null and "b/assets/images/posts/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267/Python3\345\256\236\347\216\260\347\201\253\350\275\246\347\245\250\346\237\245\350\257\242\345\267\245\345\205\267-\347\254\224\350\256\260-\345\233\2769-\350\275\246\346\254\241\344\277\241\346\201\257\350\277\224\345\233\236\347\273\223\346\236\234.png" differ
diff --git a/assets/images/posts/Search Trie/00_trie.png b/assets/images/posts/Search Trie/00_trie.png
new file mode 100644
index 00000000..56fd727b
Binary files /dev/null and b/assets/images/posts/Search Trie/00_trie.png differ
diff --git a/assets/images/posts/Search Trie/01_Trie.png b/assets/images/posts/Search Trie/01_Trie.png
new file mode 100644
index 00000000..771c655c
Binary files /dev/null and b/assets/images/posts/Search Trie/01_Trie.png differ
diff --git a/assets/images/posts/Search Trie/02_trie-detailed.png b/assets/images/posts/Search Trie/02_trie-detailed.png
new file mode 100644
index 00000000..c0e0dae3
Binary files /dev/null and b/assets/images/posts/Search Trie/02_trie-detailed.png differ
diff --git a/assets/images/posts/Single-Linked-List-Python/01_linked_list.png b/assets/images/posts/Single-Linked-List-Python/01_linked_list.png
new file mode 100644
index 00000000..f429f27c
Binary files /dev/null and b/assets/images/posts/Single-Linked-List-Python/01_linked_list.png differ
diff --git a/assets/images/posts/Single-Linked-List-Python/02_recursion_in_linkedlist.png b/assets/images/posts/Single-Linked-List-Python/02_recursion_in_linkedlist.png
new file mode 100644
index 00000000..28bf231d
Binary files /dev/null and b/assets/images/posts/Single-Linked-List-Python/02_recursion_in_linkedlist.png differ
diff --git a/assets/images/posts/Single-Linked-List-Python/03_reverse_linked_list.png b/assets/images/posts/Single-Linked-List-Python/03_reverse_linked_list.png
new file mode 100644
index 00000000..74bcde3c
Binary files /dev/null and b/assets/images/posts/Single-Linked-List-Python/03_reverse_linked_list.png differ
diff --git a/assets/images/posts/ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform/minimal-fuse-amq-ha.png b/assets/images/posts/ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform/minimal-fuse-amq-ha.png
deleted file mode 100644
index 04a3a87b..00000000
Binary files a/assets/images/posts/ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform/minimal-fuse-amq-ha.png and /dev/null differ
diff --git a/assets/images/posts/ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform/top.jpg b/assets/images/posts/ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform/top.jpg
deleted file mode 100644
index b769bfd3..00000000
Binary files a/assets/images/posts/ansible-automation-for-highly-available-jboss-fuse-amq-integration-platform/top.jpg and /dev/null differ
diff --git a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/faster_build.png b/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/faster_build.png
deleted file mode 100644
index 99b10c3f..00000000
Binary files a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/faster_build.png and /dev/null differ
diff --git a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/login_nexus.png b/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/login_nexus.png
deleted file mode 100644
index 140ec0f1..00000000
Binary files a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/login_nexus.png and /dev/null differ
diff --git a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_proxy_repo.png b/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_proxy_repo.png
deleted file mode 100644
index 9e2234ef..00000000
Binary files a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_proxy_repo.png and /dev/null differ
diff --git a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_public_group.png b/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_public_group.png
deleted file mode 100644
index af268a28..00000000
Binary files a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_public_group.png and /dev/null differ
diff --git a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_s2i.png b/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_s2i.png
deleted file mode 100644
index 8c642be7..00000000
Binary files a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/nexus_s2i.png and /dev/null differ
diff --git a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/top.jpg b/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/top.jpg
deleted file mode 100644
index 97a5e32d..00000000
Binary files a/assets/images/posts/faster-fuse-integration-service-s2i-openshift-deployments/top.jpg and /dev/null differ
diff --git a/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/selectemptyandspring.png b/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/selectemptyandspring.png
deleted file mode 100644
index 2bf3e040..00000000
Binary files a/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/selectemptyandspring.png and /dev/null differ
diff --git a/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/selectversion.png b/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/selectversion.png
deleted file mode 100644
index b776bcaa..00000000
Binary files a/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/selectversion.png and /dev/null differ
diff --git a/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/swagger.png b/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/swagger.png
deleted file mode 100644
index 59b52c0f..00000000
Binary files a/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/swagger.png and /dev/null differ
diff --git a/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/top.jpg b/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/top.jpg
deleted file mode 100644
index 5205a7d8..00000000
Binary files a/assets/images/posts/fuse-and-data-grid-in-library-mode-with-index-for-searching-and-cache-store/top.jpg and /dev/null differ
diff --git a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/diag.png b/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/diag.png
deleted file mode 100644
index 198ee8cc..00000000
Binary files a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/diag.png and /dev/null differ
diff --git a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/selectemptyandspring.png b/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/selectemptyandspring.png
deleted file mode 100644
index 2bf3e040..00000000
Binary files a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/selectemptyandspring.png and /dev/null differ
diff --git a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/selectversion.png b/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/selectversion.png
deleted file mode 100644
index b776bcaa..00000000
Binary files a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/selectversion.png and /dev/null differ
diff --git a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/swagger.png b/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/swagger.png
deleted file mode 100644
index 59b52c0f..00000000
Binary files a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/swagger.png and /dev/null differ
diff --git a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/top.jpg b/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/top.jpg
deleted file mode 100644
index 5205a7d8..00000000
Binary files a/assets/images/posts/fuse-data-grid-integration-via-hot-rod-with-persistence-and-indexing-for-advanced-queries/top.jpg and /dev/null differ
diff --git a/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/process_stream_file_http.png b/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/process_stream_file_http.png
deleted file mode 100644
index 3fb22c90..00000000
Binary files a/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/process_stream_file_http.png and /dev/null differ
diff --git a/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/process_stream_xml_http.png b/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/process_stream_xml_http.png
deleted file mode 100644
index 9417c76c..00000000
Binary files a/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/process_stream_xml_http.png and /dev/null differ
diff --git a/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/top.jpg b/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/top.jpg
deleted file mode 100644
index f7c20ffd..00000000
Binary files a/assets/images/posts/handling-large-streams-of-data-through-http-with-jboss-fuse/top.jpg and /dev/null differ
diff --git a/assets/images/posts/openshift-quickstart-in-5-minutes/addApp.png b/assets/images/posts/openshift-quickstart-in-5-minutes/addApp.png
deleted file mode 100644
index 1710a628..00000000
Binary files a/assets/images/posts/openshift-quickstart-in-5-minutes/addApp.png and /dev/null differ
diff --git a/assets/images/posts/openshift-quickstart-in-5-minutes/deployed.png b/assets/images/posts/openshift-quickstart-in-5-minutes/deployed.png
deleted file mode 100644
index b5d2bcd3..00000000
Binary files a/assets/images/posts/openshift-quickstart-in-5-minutes/deployed.png and /dev/null differ
diff --git a/assets/images/posts/openshift-quickstart-in-5-minutes/fillTemplate.png b/assets/images/posts/openshift-quickstart-in-5-minutes/fillTemplate.png
deleted file mode 100644
index c17cfcfd..00000000
Binary files a/assets/images/posts/openshift-quickstart-in-5-minutes/fillTemplate.png and /dev/null differ
diff --git a/assets/images/posts/openshift-quickstart-in-5-minutes/javaconsole.png b/assets/images/posts/openshift-quickstart-in-5-minutes/javaconsole.png
deleted file mode 100644
index f35da12c..00000000
Binary files a/assets/images/posts/openshift-quickstart-in-5-minutes/javaconsole.png and /dev/null differ
diff --git a/assets/images/posts/openshift-quickstart-in-5-minutes/logged.png b/assets/images/posts/openshift-quickstart-in-5-minutes/logged.png
deleted file mode 100644
index fe8c9864..00000000
Binary files a/assets/images/posts/openshift-quickstart-in-5-minutes/logged.png and /dev/null differ
diff --git a/assets/images/posts/openshift-quickstart-in-5-minutes/podInfo.png b/assets/images/posts/openshift-quickstart-in-5-minutes/podInfo.png
deleted file mode 100644
index 51d6a894..00000000
Binary files a/assets/images/posts/openshift-quickstart-in-5-minutes/podInfo.png and /dev/null differ
diff --git a/assets/images/posts/openshift-quickstart-in-5-minutes/runningpod.png b/assets/images/posts/openshift-quickstart-in-5-minutes/runningpod.png
deleted file mode 100644
index 58c2c60c..00000000
Binary files a/assets/images/posts/openshift-quickstart-in-5-minutes/runningpod.png and /dev/null differ
diff --git a/assets/images/posts/openshift-quickstart-in-5-minutes/top.jpg b/assets/images/posts/openshift-quickstart-in-5-minutes/top.jpg
deleted file mode 100644
index 4620f017..00000000
Binary files a/assets/images/posts/openshift-quickstart-in-5-minutes/top.jpg and /dev/null differ
diff --git a/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/os-container-jvm-app-stack.svg b/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/os-container-jvm-app-stack.svg
deleted file mode 100644
index 191e7f1b..00000000
--- a/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/os-container-jvm-app-stack.svg
+++ /dev/null
@@ -1,110 +0,0 @@
-
-
diff --git a/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/quarkus_graphics_v3_bootmem_wide_03.png b/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/quarkus_graphics_v3_bootmem_wide_03.png
deleted file mode 100644
index c0ffefa9..00000000
Binary files a/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/quarkus_graphics_v3_bootmem_wide_03.png and /dev/null differ
diff --git a/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/scaling-up.svg b/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/scaling-up.svg
deleted file mode 100644
index 5d094c5e..00000000
--- a/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/scaling-up.svg
+++ /dev/null
@@ -1,220 +0,0 @@
-
-
diff --git a/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/top.jpg b/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/top.jpg
deleted file mode 100644
index 5a2a8007..00000000
Binary files a/assets/images/posts/quarkus-camel-first-impressions-on-a-game-changing-framework/top.jpg and /dev/null differ
diff --git a/assets/images/posts/quarkus-kafka-camel-servlet-wiring-it-together/top.jpg b/assets/images/posts/quarkus-kafka-camel-servlet-wiring-it-together/top.jpg
deleted file mode 100644
index 27b62a85..00000000
Binary files a/assets/images/posts/quarkus-kafka-camel-servlet-wiring-it-together/top.jpg and /dev/null differ
diff --git a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/choose-simple-project.png b/assets/images/posts/quickstart-rest-microservice-with-spring-boot/choose-simple-project.png
deleted file mode 100644
index 633c7aaa..00000000
Binary files a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/choose-simple-project.png and /dev/null differ
diff --git a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/consume.png b/assets/images/posts/quickstart-rest-microservice-with-spring-boot/consume.png
deleted file mode 100644
index db2d028c..00000000
Binary files a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/consume.png and /dev/null differ
diff --git a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/create-new-application-class.png b/assets/images/posts/quickstart-rest-microservice-with-spring-boot/create-new-application-class.png
deleted file mode 100644
index 374172ca..00000000
Binary files a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/create-new-application-class.png and /dev/null differ
diff --git a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/create-new-maven-project.png b/assets/images/posts/quickstart-rest-microservice-with-spring-boot/create-new-maven-project.png
deleted file mode 100644
index baa14832..00000000
Binary files a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/create-new-maven-project.png and /dev/null differ
diff --git a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/fill-project-infos.png b/assets/images/posts/quickstart-rest-microservice-with-spring-boot/fill-project-infos.png
deleted file mode 100644
index 2e41c13a..00000000
Binary files a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/fill-project-infos.png and /dev/null differ
diff --git a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/top.jpg b/assets/images/posts/quickstart-rest-microservice-with-spring-boot/top.jpg
deleted file mode 100644
index 146c7726..00000000
Binary files a/assets/images/posts/quickstart-rest-microservice-with-spring-boot/top.jpg and /dev/null differ
diff --git a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/brokernetworkconcentrator.png b/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/brokernetworkconcentrator.png
deleted file mode 100644
index f4ddb66a..00000000
Binary files a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/brokernetworkconcentrator.png and /dev/null differ
diff --git a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/distributeload.png b/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/distributeload.png
deleted file mode 100644
index f0483a75..00000000
Binary files a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/distributeload.png and /dev/null differ
diff --git a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/examplenetworkconfig.png b/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/examplenetworkconfig.png
deleted file mode 100644
index 8952e995..00000000
Binary files a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/examplenetworkconfig.png and /dev/null differ
diff --git a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/scaledown.png b/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/scaledown.png
deleted file mode 100644
index 41088ad8..00000000
Binary files a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/scaledown.png and /dev/null differ
diff --git a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/top.jpg b/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/top.jpg
deleted file mode 100644
index 87fb5e7d..00000000
Binary files a/assets/images/posts/scalable-network-active-mq-brokers-handing-massive-connections/top.jpg and /dev/null differ
diff --git a/assets/images/top.jpg b/assets/images/top.jpg
index ed3527b8..295b6815 100644
Binary files a/assets/images/top.jpg and b/assets/images/top.jpg differ
diff --git a/favicon.ico b/favicon.ico
new file mode 100644
index 00000000..18177b57
Binary files /dev/null and b/favicon.ico differ
diff --git a/icon.png b/icon.png
deleted file mode 100644
index db6f81c4..00000000
Binary files a/icon.png and /dev/null differ
diff --git a/index.html b/index.html
index 8f779674..e10d1adb 100644
--- a/index.html
+++ b/index.html
@@ -1,7 +1,7 @@
---
layout: default
-title: Alain's Tech Blog
-description: Sharing about Application Development, Integration, Cloud, Big Data and Digital Transformation
+title: Weiwei's Tech Blog
+description: Sharing about Software Development, Algorithms and others
comments: true
---
diff --git a/readme.md b/readme.md
new file mode 100644
index 00000000..8999b9db
--- /dev/null
+++ b/readme.md
@@ -0,0 +1,41 @@
+---
+layout: default
+title: My Personal Github Page
+author: Weiwei Cao
+date: 2020-04-27
+---
+
+# My Personal Github Page
+
+This template is Based on [alainpham.github.io](https://github.com/alainpham/alainpham.github.io). Thanks to the author! Below is the introduction of this respository
+
+## Quickstart
+
+- `fork` the repository into your own github and modify the `settings` of the repository, please check here for [Github Pages](https://pages.github.com/)
+- modify the file `_config.yml` to change the configurations
+- Edit the file `index.md` to make personal homepage
+- Delete all the files at `_posts/` and upload your own blogs
+- All above can be done using `git` command, more details see *[jekyll](https://jekyllrb.com/)*
+
+## Structure
+
+### For the content：
+
+- `_data`: modify the`settings.yml` file to configure the navigation menu
+- `_drafts`: for the posts that you're wokring on and don't want to publish
+- `_posts`: for posted blogs, you typically write posts in `Markdown`, `HTML` is also supported
+- `pages`: for webpages, such as a home page, 404 page and other project pages.
+- `assets/imagines`: all imagines used by the github page, including blogs and head banner
+- `robots.txt`: for SEO usage
+- `icon.png`: a browser tab icon (aka favicon)
+- `index.html`: your homepage
+- `404.html`: 404 page
+
+### For the page layout：
+
+- `_includes`：page components, including header, footer, analytics and 3rd party plugins components
+- `_layouts`：webpage templates
+- `assets/css`：webpage css files
+- `assets/fonts`：fonts
+- `assets/js`：javascripts
+- `_config.yml`：configurations
\ No newline at end of file
diff --git a/robots.txt b/robots.txt
index ac3dc9d7..c0a0b296 100644
--- a/robots.txt
+++ b/robots.txt
@@ -2,4 +2,4 @@
# www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
User-agent: *
-Sitemap: http://alainpham.github.io/sitemap.xml
\ No newline at end of file
+Sitemap: http://cao-weiwei.github.io/sitemap.xml
diff --git a/thumbnail.png b/thumbnail.png
deleted file mode 100644
index b693f7d8..00000000
Binary files a/thumbnail.png and /dev/null differ