diff --git a/graphs/graph_resources.md b/graphs/graph_resources.md deleted file mode 100644 index fb581ef..0000000 --- a/graphs/graph_resources.md +++ /dev/null @@ -1,32 +0,0 @@ - -## Glossary -1. **vertex (a.k.a. node):** used to represent one data point -2. **edge:** connections between pairs of vertices -3. **neighbor:** a neighbor node refers to a node that is directly connected to another node by an edge -4. **directed graph:** a graph that in which all edges are associated with a direction -5. **undirected graph:** a graph that in which all edges have no directions -6. **path:** is a sequence of vertices connected by edges -7. **cycle:** a path whose first and last vertices are the same -8. **cyclic graph:** a graph in which at least one cycle exists -9. **acyclic graph:** a graph in which no cycle exists -10. **adjacency list:** an approach to represent graphs in which each node stores a list of its adjacent vertices -11. **edge set/list:** an approach to represent graphs in which a graph as a collection of all its edges -12. **adjacency matrix:** an approach to represent graphs in which graph with _n_ nodes as a _n_ by _n_ boolean matrix, in which matrix\[_u_\]\[_v_\] is set to true if an edge exists from node _u_ to node _v_. -13. **breadth first search:** pick an arbitrary node as the root and explore each of its neighbors before visiting their children -14. **depth first search:** start with an arbitrary node as a root and explore each neighbor fully before exploring the next one -15. **topological sort:** an ordering of nodes for a directed acyclic graph (DAG) such that for every directed edge _uv_ from vertex _u_ to vertex _v_, _u_ comes before _v_ in the ordering. -16. **sink nodes:** in a DAG, a sink node has no outgoing edges -17. **source nodes:** in a DAG, a source node only has outgoing edges -18. **directed acylic graph (DAG):** a directed graph in which there are *no* cycles - - -## Extra graph algorithms -**NOTE**: This section covers algorithms that will generally not come up in interviews. -### Union find, disjoint sets - -### Shortest paths algorithms - -### Minimum spanning tree algorithms - - -## Resources diff --git a/graphs/graph_traversals.md b/graphs/graph_traversals.md index 8c5fc87..a959041 100644 --- a/graphs/graph_traversals.md +++ b/graphs/graph_traversals.md @@ -1,10 +1,9 @@ -# Graph traversals These traversal algorithms are conceptually the same as the ones introduced in the tree section. ## Depth first search In a **depth first search**, we start with an arbitrary node as a root and explore each neighbor fully before exploring the next one. - + **Implementation:** ```python @@ -39,9 +38,9 @@ def depth_first_search(graph, start): * [Detect a cycle in a graph](https://www.geeksforgeeks.org/detect-cycle-in-a-graph/) ## Breadth first search -In **breadth first search**, we pick an arbitrary node as the root and explore each of its neighbors before visiting their children. Breadth first search is the better of the two algorithms at finding the shortest path between two nodes. +In **breadth first search**, we pick an arbitrary node as the root and explore each of its neighbors before visiting their children. Breadth first search is the better suited at finding the shortest path between two nodes. - + **Implementation:** ```python @@ -74,9 +73,12 @@ def breadth_first_search(graph, start): **Example interview question using BFS:** +* [Clone an undirected graph](https://www.geeksforgeeks.org/clone-an-undirected-graph/) + **Runtime**: O(V + E) ## Key Takeaways: * DFS is better for analyzing structure of graphs (ex. looking for cycles) * BFS is better for optimization (ex. shortetest path algorithms) + diff --git a/graphs/intro_to_graphs.md b/graphs/intro_to_graphs.md index 4cbb4c0..9cb1106 100644 --- a/graphs/intro_to_graphs.md +++ b/graphs/intro_to_graphs.md @@ -1,60 +1,59 @@ -# Introduction to Graphs -# Introduction +## Introduction Graphs are one of the most prevalent data structures in computer science. It's a powerful data structure that's utilized to represent relationships between different types of data. In a graph, each data point is stored as a **node** and each relationship between those data points is represented by an **edge**. For example, a social network is considered a graph in which each person is considered a node and a friendship between two people is considered an edge. Graphs are best utilized for problems in which there are binary relationships between objects. Once a problem can be represented as a graph, the problem can generally be solved based off of one of the key graph algorithms. For interviews, it is vital to know how to implement a graph, basic graph traversals (BFS, DFS) and how to topologically sort the graph. -# Graph Terminology -## Graph components +## Graph Terminology +### Graph components Graphs consist of a set of.. * **vertices**, which are also referred to as nodes * Nodes that are directly connected by an edge are commonly referred to as **neighbors**. * **edges**, connections between pairs of vertices - + -## Graph types +### Graph types -### Directed & undirected graphs +#### Directed & undirected graphs A **directed** graph is a graph that in which all edges are associated with a direction. An example of a directed edge would be a one way street. An **undirected** graph is a graph in which all edges do not have a direction. An example of this would be a friendship! - + -### Cyclic & acyclic graphs +#### Cyclic & acyclic graphs Before going over the what cyclic and acyclic graphs are, there are two key terms to cover: **path** and **cycle**. A **path** is a sequence of vertices connected by edges and a **cycle** a path whose first and last vertices are the same. A **cyclic** graph means that there contains a least one cycle within the graph. An **acyclic** graph has no cycles within it. - + A commonly used phrase when referring to graphs is a **directed acylic graph (DAG)**, which is a directed graph in which there are *no* cycles. In a DAG, these two terms are commonly used to denote nodes with special properties: * **Sink** nodes have no outgoing edges, only incoming edges * **Source** nodes only have outgoing edges, no incoming edges -# Graph representations -## Adjacency lists +## Graph representations +### Adjacency lists **Adjacency list** is the most common way to represent graphs. With this approach of representing a graph, each node stores a list of its adjacent vertices. For undirected graphs, each edge from _u_ to _v_ would be stored twice: once in _u_'s list of neighbors and once in _v_'s list of neighbors. - + -## Edge sets/ lists +### Edge sets/ lists An **edge set** simply represents a graph as a collection of all its edges. - + -## Adjacency matrix +### Adjacency matrix An **adjacency matrix** represents a graph with _n_ nodes as a _n_ by _n_ boolean matrix, in which matrix[_u_][_v_] is set to true if an edge exists from node _u_ to node _v_. - + The representation of a graph is efficient for checking if an edge exists between a pair of vertices. However, it may be less efficient for search algorithms because it requires iterating through all the nodes in the graph to identify a node's neighbors. -## Runtime Analysis +### Runtime Analysis Below is a chart of the most common graph operations and their runtimes for each of the graph representations. In the chart below, _V_ represents the number of verticies in the graph and _E_ represents the number of edges in the graph. | Representation | Getting all adjacent edges for a vertex| Traversing entire graph | hasEdge(u, v) | Space | @@ -64,3 +63,34 @@ Below is a chart of the most common graph operations and their runtimes for each | **Adjacency List** | O(1) | O(V + E) | O(max number of edges a vertex has) | O(E + V) Credit: [UC Berkeley data structures course](https://docs.google.com/presentation/d/1GOOt1Ierm9jJFq9o26uRW20GdU6E5hrAZvsoQIreJew) + +## Glossary +1. **vertex (node):** used to represent a single data point +2. **edge:** a connection between a pair of vertices +3. **neighbor:** a neighbor node is a node that is directly connected to another node by an edge +4. **directed graph:** a graph in which all edges have direction +5. **undirected graph:** a graph in which all edges have no direction +6. **path:** a sequence of vertices connected by edges +7. **cycle:** a paththat begins and ends at the same vertex +8. **cyclic graph:** a graph which contains at least one cycle +9. **acyclic graph:** a graph whichdoes not contain a cycle +10. **adjacency list:** an approach to representing graphs in which each node stores a list of its adjacent vertices +11. **edge set/list:** an approach to representing graphs in which a graph is a collection of all its edges +12. **adjacency matrix:** an approach to representing graphs in which a graph with _n_ nodes is storeed as an _n_ by _n_ boolean matrix, where matrix\[_u_\]\[_v_\] is true if an edge exists between node _u_ to node _v_. +13. **sink nodes:** in a DAG, a sink node has no outgoing edges +14. **source nodes:** in a DAG, a source node only has outgoing edges +15. **directed acylic graph (DAG):** a directed graph in which there are *no* cycles + +## Extra graph algorithms +**NOTE**: This section covers algorithms that will generally not come up in interviews. +### Union find, disjoint sets +* Guide: https://www.hackerearth.com/practice/notes/disjoint-set-union-union-find/ +* Interview question bank: https://leetcode.com/tag/union-find/ + +### Shortest paths algorithms +* Guide: https://www.hackerearth.com/practice/algorithms/graphs/shortest-path-algorithms/tutorial/ +* Interview question bank: https://www.hackerearth.com/practice/algorithms/graphs/shortest-path-algorithms/practice-problems/ + +### Minimum spanning tree algorithms +* Guide: https://www.hackerearth.com/practice/algorithms/graphs/minimum-spanning-tree/tutorial/ +* Interview question bank: https://www.hackerearth.com/practice/algorithms/graphs/minimum-spanning-tree/practice-problems/ diff --git a/graphs/top_sort.md b/graphs/top_sort.md deleted file mode 100644 index 8929363..0000000 --- a/graphs/top_sort.md +++ /dev/null @@ -1,35 +0,0 @@ -# Topological Sorting -Aside from DFS and BFS, the most common graph concept that interviews will test is topological sorting. The objective of this algorithm is to produce an ordering of nodes in a directed graph such that the direction of nodes is respected. - -A **topological sort** is an ordering of nodes for a directed acyclic graph (DAG) such that for every directed edge _uv_ from vertex _u_ to vertex _v_, _u_ comes before _v_ in the ordering. - -## Example -An application of this algorithm would be trying to order a sequence of tasks given their dependencies on other tasks. In this application, there is an directed edge from _u_ to _v_ if task _u_ must be completed before task _v_ can start. For example, when cooking, we need to turn on the oven (task _u_) before we can bake the cookies (task _v_). - - - -## Implementation: -The algorithm behind how to do this is simply a modification of DFS. - -### Graph with no cycles -```python -from collections import deque - -def top_sort(graph): - sorted_nodes, visited = deque(), set() - for node in graph.keys(): - if node not in visited: - dfs(graph, node, visited, sorted_nodes) - return list(sorted_nodes) - - -def dfs(graph, start_node, visited, sorted\_nodes): - visited.add(start_node) - if start_node in graph: - neighbors = graph[start_node] - for neighbor in neighbors: - if neighbor not in visited: - dfs(graph, neighbor, visited, sorted_nodes) - sorted_nodes.appendleft(start_node) -``` - diff --git a/graphs/topological_sort.md b/graphs/topological_sort.md new file mode 100644 index 0000000..3fdf8b4 --- /dev/null +++ b/graphs/topological_sort.md @@ -0,0 +1,34 @@ +## Introduction to Topological Sorting +Aside from DFS and BFS, the most common graph concept that interviews will test is topological sorting. Topological sorting produces a linear ordering of nodes in a directed graph such that the direction of edges is respected. + +A **topological sort** is an ordering of nodes for a directed acyclic graph (DAG) such that for every directed edge _uv_ from vertex _u_ to vertex _v_, _u_ comes before _v_ in the ordering. + +## Example +An application of this algorithm is ordering a sequence of tasks given their dependencies on other tasks. In this application, there is an directed edge from _u_ to _v_ if task _u_ must be completed before task _v_ can start. For example, when cooking, we need to turn on the oven (task _u_) before we can bake the cookies (task _v_). + + + +## Implementation: +Topological sort is simply a modification of DFS. Topological sort simply involves running DFS on an entire graph and adding each node to the global ordering of nodes only after all of a node's children are visited. This ensures that parent nodes will be ordered before their child nodes honoring the forward direction of edges in the ordering. + +### Graph with no cycles +```python +from collections import deque + +def top_sort(graph): + sorted_nodes, visited = deque(), set() + for node in graph.keys(): + if node not in visited: + dfs(graph, node, visited, sorted_nodes) + return list(sorted_nodes) + + +def dfs(graph, start_node, visited, sorted_nodes): + visited.add(start_node) + if start_node in graph: + neighbors = graph[start_node] + for neighbor in neighbors: + if neighbor not in visited: + dfs(graph, neighbor, visited, sorted_nodes) + sorted_nodes.appendleft(start_node) +``` diff --git a/hash_tables/figures/hash_collision.png b/hash_tables/figures/hash_collision.png new file mode 100644 index 0000000..18ceda2 Binary files /dev/null and b/hash_tables/figures/hash_collision.png differ diff --git a/hash_tables/figures/hash_table.png b/hash_tables/figures/hash_table.png new file mode 100644 index 0000000..94a052a Binary files /dev/null and b/hash_tables/figures/hash_table.png differ diff --git a/hash_tables/hash_tables.md b/hash_tables/hash_tables.md new file mode 100644 index 0000000..9bb3629 --- /dev/null +++ b/hash_tables/hash_tables.md @@ -0,0 +1,58 @@ +## Introduction +**Hash tables** are one of the most common and useful data structures in both interviews and real life. The biggest advantage of hash tables is that they provide quick key-to-value lookup. + +One use of a hash table would be to implement a phone book. In this scenario, the key is a name and the value is a phone number. + +```python +address_book = {} +address_book["Bob"] = "111-222-3333" +address_book["Alice"] = "444-555-6666" +address_book.get("Bob") +'111-222-3333' +``` + +Hash tables are very efficient. Operations such as insertion, deletion, and get take, on average, constant time. + +## How it works: +### Hash Codes +Internally,a hash table stores its values in an array. A special **hash function** is utilized to convert each key into a code, which is then converted into an index into the underlying array. This hash function has a hard requirement to return the same hash code for equal keys. + +Hash function generate a code, known as the **hash code**. Each data type will have its own hash function that generates its hash code differently. It is important to remember that a hash code is not equivalent to the index in the underlying array storage structure--there are often more hash codes than indices in the underlying array. + +**Figure below**: Internals of a hash table +![](https://i.imgur.com/bEIWPaQ.png) + + +**Note:** When utilizing a hash table with a class you've created, be sure that the hash function for that object type operates as you would expect. If two objects are equivalent, you should ensure that their hash codes are the same. + + +### Collisions +If the hash function is implemented well, inputted objects will be distributed evenly across the array indices. However, the number of hash codes is often greater than the size of the underlying array, so some keys will be assigned the same index. When two keys are matched to the same index, this is called a **collision**. There are several different ways of addressing collisions. + +There are multiple approaches to dealing with collisions. The most common one is simply to store all the objects that get assigned to the same index in a linked list. In this scenario, instead of simply storing the value at that index, the linked list must contain both the entire key and the value in pairs instead of just the value, so that the values can be uniquely tied to a key. + +**Figure below**: Hash Collision +![](https://i.imgur.com/ZqF2crs.png) + +For more information about other ways to address hash collisions, see the [hash tables Wikipedia page](https://en.wikipedia.org/wiki/Hash_table#Open_addressing) for descriptions of several different approaches. If you have time, I'd encourage you to learn about open addressing, the other common hash collision method. + +**Note on updating keys in the hash table**: Be wary of updating a key object that's present in the hash table. Once it's updated, lookup will no longer work for that key. Instead, if a key needs to be updated, remove it, update the key, then re-insert the key into the table. + +## Runtimes +For interviews, it is generally assumed that the hash table is a well formed one, with few collisions. However, in the worst case scenario, all operations can take linear time. When a hash table distributes the objects poorly, all the objects may hash to the same index in the underlying array and any operation would require going through all of the previous entries. + +| | Lookup | Insert | Delete | +| -------- | -------- | -------- | -------- | +| Best Case | O(1) | O(1) | O(1) | +| Worst Case| O(n) | O(n) | O(n) | + + +## Key takeaways +* Hash tables are an extremely useful data structure-- keep them in mind as a tool for most interview questions +* Hash tables have the best performance for lookups, inserts, and deletions. All three operations on average take O(1) time. + +## Resources +To get a more thorough understanding on the internals of hash tables, here are a few links to helpful resources. +* Hash table guide with helpful diagrams: https://medium.com/basecs/hashing-out-hash-functions-ea5dd8beb4dd +* Princeton Coursera video lecture series: https://www.coursera.org/learn/algorithms-part1/lecture/CMLqa/hash-tables +* HackerRank video (short summary): https://youtu.be/shs0KM3wKv8 diff --git a/heaps/figures/bubbling_down.png b/heaps/figures/bubbling_down.png new file mode 100644 index 0000000..e39afc3 Binary files /dev/null and b/heaps/figures/bubbling_down.png differ diff --git a/heaps/figures/bubbling_up.png b/heaps/figures/bubbling_up.png new file mode 100644 index 0000000..3e98d3f Binary files /dev/null and b/heaps/figures/bubbling_up.png differ diff --git a/heaps/figures/heap_as_array.png b/heaps/figures/heap_as_array.png new file mode 100644 index 0000000..de6fb77 Binary files /dev/null and b/heaps/figures/heap_as_array.png differ diff --git a/heaps/figures/min_max_heap.png b/heaps/figures/min_max_heap.png new file mode 100644 index 0000000..285d7be Binary files /dev/null and b/heaps/figures/min_max_heap.png differ diff --git a/heaps/heaps.md b/heaps/heaps.md new file mode 100644 index 0000000..6d9f5e9 --- /dev/null +++ b/heaps/heaps.md @@ -0,0 +1,77 @@ +## Introduction +Heaps are an often overlooked data structure, but come up quite often in interview problems. **Heaps** are special tree based data structures that satisfy two properties: + +1. All nodes are ordered in a specific way, depending on the type of heap. There are two types of heaps: min heaps and max heaps. + * In **min heaps**, the root node contains the smallest element and all the nodes in the heap contain elements that are less than their child nodes. + * In **max heaps**, the root node contains the largest element and all the nodes in the heap contain elements that are greater than their child nodes. + +2. It is a complete binary tree. A **binary tree's** nodes will have at most two children: a left child, and right child. A heap is a complete binary tree, which means that it fills each level entirely except the last level. Another way of thinking about this is that all the nodes in one level will have children before any of those nodes will have grandchildren. + + + +## Heap Operations +In order to understand the runtimes of heap operations, it is vital to understand how insertion and deletion work within a heap. + +### Insertion +When a new element is inserted into a heap, it is added in the next empty spot in the heap, in the left most position in the last level of the heap, in order to maintain the full shape of the heap. However, this new item may violate the other key property of the heap, its ordering. + +In a min heap, if the parent of the new element is greater than it, it gets swapped with the parent. This element keeps getting **bubbled up** in the tree until it either reaches the root of the heap or it has been placed in the right order. This same process applies to max heaps as well, but the check to ensure that the node is in the proper position is that the parent node is greater than the new node. + + + +### Removal +When removing from a heap, the root node is always removed. Then, the last element, the leftmost node in the last level of the heap, is removed and set as the root. This removal process retains the heap shape, but this new ordering may violate the proper ordering of the heap. + +In a min heap, if either one of the new element's children are less than their parent, the new element is swapped with the smaller of the two children. This element keeps getting **bubbled down** in the tree until it either reaches the last level of the heap or it has been placed in the right position. The same process applies to max heaps as well, but the ordering is such that the children are both greater than the current node. + + + +### Building a heap from a list +One approach to building a heap from a list of N elements is starting with an empty heap and adding each item from a list, one at a time. This approach takes O(N log N) time because it performs N insertions, each of which takes log N time. However, this approach is suboptimal and the optimal approach of building a heap from N items only takes O(N) time! + +The math and implementation behind this optimization are a bit complex, but are explained well in the [Wikipedia page](https://en.wikipedia.org/wiki/Binary_heap#Building_a_heap). Itâ™s a good idea to get a general understanding of the optimization. + +## Implementation +Surprisingly, this complex data structure can be represented using an array! Given that the root node will always be either the least or greatest element in the heap, we can place this element as the first element in the array. This underlying array is then filled up by going through each level of the heap, from left to right, top to bottom. + +With the guarantee of fullness and the binary tree property of the heap, we can easily calculate the indices of the children and parents of each node using these formulas: +* Parent: (current index - 2) / 2 +* Left child: (current index * 2) + 1 +* Right child: (current index * 2) + 2 + + + +These calculations enable it to easily implement the insertion and removal procedures within the array. + +## Runtimes +In the worst case scenario, the swapping procedure for insertions and deletions will move the element through the height of the heap. Because heaps are binary trees that are guaranteed to be as complete as possible, the number of levels in the heap will be log n. + +| Operation | Runtime | +| ------------------------------------| -------- | +| Reading largest or smallest element | O(1) | +| Insertion | O(log n) | +| Deletion | O(log n) | +| Creating a heap from a list | O(n) | + +## Key takeaways +* Heaps are especially useful when for getting the largest or smallest elements, and in situations where you don't care about fast lookup, delete, or search. +* Heaps are especially useful for questions that involve getting the x-largest or x-smallest elements of some data set. +* Building a heap only takes O(n) time, so you can potentially optimize a solution by building a heap from a list instead of running insertion n times to create the heap. + +## Glossary +**Binary tree**: a tree that has at most two children. +**Bubble down**: the process of moving an element down within the heap by swapping it with one of its children until it is placed in the proper position that satisfies the heap ordering. +**Bubble up**: the process of moving an element up within the heap by swapping it with its parent until it is placed in the proper position that satisfies the heap ordering. +**Max heap**: tree based structure in which the root node contains the largest element and all the nodes in the heap contain elements that are greater than their child nodes. +**Min heap**: tree based structure in which the root node contains the smallest element and all the nodes in the heap contain elements that are less than their child nodes. + +## Example problems +* [Top K frequent words](https://leetcode.com/problems/top-k-frequent-words/description/) +* [Find K pairs with smallest sums](https://leetcode.com/problems/find-k-pairs-with-smallest-sums/description/) +* [Implementing a heap](http://interactivepython.org/courselib/static/pythonds/Trees/BinaryHeapImplementation.html) + +## Resources +To get a more thorough understanding of the internals of heaps, here are a few links to helpful resources. +* Guide on heaps, with helpful diagrams: https://medium.com/basecs/learning-to-love-heaps-cef2b273a238 +* Princeton Coursera video lecture series: https://www.coursera.org/learn/algorithms-part1/lecture/Uzwy6/binary-heaps +* HackerRank video: https://youtu.be/t0Cq6tVNRBA diff --git a/stacks_queues/figures/queues.png b/stacks_queues/figures/queues.png new file mode 100644 index 0000000..463724f Binary files /dev/null and b/stacks_queues/figures/queues.png differ diff --git a/stacks_queues/figures/stacks.png b/stacks_queues/figures/stacks.png new file mode 100644 index 0000000..bee1d1f Binary files /dev/null and b/stacks_queues/figures/stacks.png differ diff --git a/stacks_queues/stacks_queues.md b/stacks_queues/stacks_queues.md new file mode 100644 index 0000000..124685b --- /dev/null +++ b/stacks_queues/stacks_queues.md @@ -0,0 +1,48 @@ +## Introduction + +Stacks and queues are foundational data structures that are useful for problems that rely on adding and removing elements in a particular order. It's important to be comfortable with these two data structures. + +## Stacks +A **stack** stores objects such that the most recently added objects are the first ones to be removed (LIFO: last in, first out). An example to help you remember the mechanics of a stack is to associate it with stacks in real life. In a stack of plates, the ones placed on top will be the first ones removed! + +![](https://i.imgur.com/qMSmxsa.png) + +It's important to know the common operations of a stack. The two key stack operations are: +1. pop(): removing an item from the stack in last in, first out order (LIFO) +2. push(item): adding an item (to the top of the stack) + + +## Queues +A **queue** stores objects such that the objects added earliest are the first ones to be removed (FIFO: first in first out). An example to help you remember the mechanics of a queue is to associate it with queues in real life. In a queue of people waiting to get a seat in a restaurant, the first people in line (in the queue) will be the first to get a table. + +![](https://i.imgur.com/NKuZd0s.png) + +It's important to know the common operations associated with a queue. The two important queue operations are: +1. dequeue(): removing an item from the queue in first in, first out order (FIFO) +2. enqueue(item): adding an item (to the back of the queue) + +## Key takeaways +* Stacks are very useful for their backtracking features. For example, parsing questions tend to use stacks because of their LIFO property. +* Stacks can be used to implement recursive solutions iteratively. +* Queues are useful when the ordering of the data matters because they preserve the original ordering. For example, queues are used for caching. + +## Example problems +* Stacks: + * [Implement a queue with stacks](https://leetcode.com/problems/implement-queue-using-stacks/description/) + +* Queues: + * [LRU Cache](https://leetcode.com/problems/lru-cache/) + +## Resources +### Guides +* [Overview of stacks and queues with applications](https://www.cs.cmu.edu/~adamchik/15-121/lectures/Stacks%20and%20Queues/Stacks%20and%20Queues.html) +* [In-depth stacks guide](https://medium.com/basecs/stacks-and-overflows-dbcf7854dc67) +* [In-depth queues guide](https://medium.com/basecs/to-queue-or-not-to-queue-2653bcde5b04) + +### Libraries +* [Java Queue library](https://docs.oracle.com/javase/7/docs/api/java/util/Queue.html) +* [Java Stack library]( +https://docs.oracle.com/javase/7/docs/api/java/util/Stack.html) +* [Python queue library]( +https://docs.python.org/2/tutorial/datastructures.html#using-lists-as-queues) +* [Python stack library](https://docs.python.org/2/tutorial/datastructures.html#using-lists-as-stacks) diff --git a/strings_arrays/binary_search.md b/strings_arrays/binary_search.md new file mode 100644 index 0000000..840fa48 --- /dev/null +++ b/strings_arrays/binary_search.md @@ -0,0 +1,72 @@ +### Introduction + +Binary search is a technique for efficiently locating an element in a sorted list. Searching for an element can done naively in **O(n)** time by checking every element in the list, but binary search's optimization speeds it up to **O(log n)**. Binary search is a great tool to keep in mind for array problems. + +Algorithm +------------------ +In binary search, you are provided a sorted list of numbers and a key. The desired output of a binary search is the index of the key in the sorted list, if the key is in the list, or ```None``` otherwise. + +Binary search is a recursive algorithm. From a high-level perspective, we examine the middle element of the list, which determines whether to terminate the algorithm (found the key), recursively search the left half of the list (middle element value > key), or recursively search the right half of the list (middle element value < key). +``` +def binary_search(nums, key): + if nums is empty: + return None + if middle element is equal to key: + return middle index + if middle element is greater than key: + binary search left half of nums + if middle element is less than + binary search right half of nums +``` + +There are two canonical ways of implementing binary search: recursive and iterative. Both solutions utilizes two pointers that keep track of the portion of the list we are searching. + +### Recursive Binary Search + +The recursive approach utilizes a helper function to keep track of pointers to the section of the list we are currently examining. The search either terminates when we find the key or if the two pointers meet. + +```python +def binary_search(nums, key): + return binary_search_helper(nums, key, 0, len(nums)) + +def binary_search_helper(nums, key, start_idx, end_idx): + middle_idx = (start_idx + end_idx) // 2 + if start_idx == end_idx: + return None + if nums[middle_idx] > key: + return binary_search_helper(nums, key, start_idx, middle_idx) + elif nums[middle_idx] < key: + return binary_search_helper(nums, key, middle_idx + 1, end_idx) + else: + return middle_idx +``` + +### Iterative Binary Search + +The iterative approach manually keeps track of the section of the list we are examining using the two-pointer technique. The search either terminates when we find the key, or the two pointers meet. +```python +def binary_search(nums, key): + left_idx, right_idx = 0, len(nums) + while right_idx > left_idx: + middle_idx = (left_idx + right_idx) // 2 + if nums[middle_idx] > key: + right_idx = middle_idx + elif nums[middle_idx] < key: + left_idx = middle_idx + 1 + else: + return middle_idx + return None +``` + +## Runtime and Space Complexity + +Binary search has **O(log n)** time complexity because each iteration decreases the size of the list by a factor of 2. Its space complexity is constant because we only need to maintain two pointers. Even the recursive solution has constant space with [tail call optimization](https://en.wikipedia.org/wiki/Tail_call). + +## Example problems +* [Search insert position](https://leetcode.com/problems/search-insert-position/description/) +* [Search in a 2D matrix](https://leetcode.com/problems/search-a-2d-matrix/description/) + +## Video walkthrough +* [HackerRank binary search video](https://www.youtube.com/watch?v=P3YID7liBug) +* [Question walkthrough: Search a 2D matrix](https://www.youtube.com/playlist?list=PL7zKQzeqjecINi-_8CmiFLMLCCxjIHBPj) +* [Question walkthrough: Ice Cream Parlor](https://youtu.be/Ifwf3DBN1sc) diff --git a/strings_arrays/sorting.md b/strings_arrays/sorting.md new file mode 100644 index 0000000..a96f751 --- /dev/null +++ b/strings_arrays/sorting.md @@ -0,0 +1,147 @@ +Sorting is a fundamental tool for tackling problems, and is often utilized to help simplify problems. + +There are several different sorting algorithms, each with different tradeoffs. In this guide, we will cover several well-known sorting algorithms along with when they are useful. + +We will describe merge sort and quick sort in detail and the remainder of the featured sorting algorithms at a high level. + +## Terminology +Two commonly used terms in sorting are: + +1. **in-place sort**: modifies the input list and does not return a new list +2. **stable sort**: retains the order of duplicate elements after the sort ([3, 2, 4, **2**] -> [2, **2**, 3, 4]) + +## Merge sort +**Merge sort** is perhaps the simplest sort to implement and has very consistent behavior. It adopts a divide-and-conquer strategy: recursively sort each half of the list, and then perform an O(n) merging operation to create a fully sorted list. + +### Implementation + +The key operation in merge sort is `merge`, which is a function that takes two sorted lists and returns a single sorted list composed of elements of the combined lists. +```python +def merge(list1, list2): + if len(list1) == 0: + return list2 + if len(list2) == 0: + return list1 + if list1[0] < list2[0]: + return [list1[0]] + merge(list1[1:], list2) + else: + return [list2[0]] + merge(list1, list2[1:]) +``` +This is a recursive implementation of `merge`, but an iterative implementation will also work. + +Given this `merge` operation, writing merge sort is quite simple. + +```python +def merge_sort(nums): + if len(nums) <= 1: + return nums + middle_idx = len(nums) // 2 + left_sorted = merge_sort(nums[:middle_idx]) + right_sorted = merge_sort(nums[middle_idx:]) + return merge(left_sorted, right_sorted) +``` + +### Runtime +Merge sort is a recursive, divide and conquer algorithm. It takes O(log n) recursive merge sorts and each merge is O(n) time, so we have a final runtime of O(n log n) for merge sort. Its behavior is consistent regardless of the input list (its worst case and best case take the same amount of time). + +**Summary** + +| Worst case | Best case | Stable | In-place| +|:----------:|:---------:|:------:|:-------:| +| O(n log n) | O(n log n) | ✅ | ❌ | + +## Quick sort + +**Quick sort** is also a divide and conquer strategy, but uses a two-pointer swapping technique instead of `merge`. The core idea of quick sort is selecting a "pivot" element in the list (typically the middle element), and swapping elements in the list such that everything left of the pivot is less than it, and everything right of the pivot is greater. We call this operation `partition`. Quick sort is notable for its ability to sort efficiently in-place. + +```python +def partition(nums, left_idx, right_idx): + pivot = nums[left_idx] + while True: + while nums[left_idx] < pivot and left_idx <= right_idx: + left_idx += 1 + while nums[right_idx] > pivot and right_idx >= left_idx: + right_idx -= 1 + if left_idx >= right_idx: + return right_idx + nums[left_idx], nums[right_idx] = nums[right_idx], nums[left_idx] + left_idx += 1 + right_idx -= 1 +``` +The partition function modifies `nums` in-place and requires no extra memory. It also takes O(n) time worst case to fully partition a list. + +```python +def quick_sort_helper(nums, left_idx, right_idx): + if left_idx >= right_idx: + return + pivot_idx = partition(nums, left_idx, right_idx) + if left_idx < pivot_idx - 1: + quick_sort_helper(nums, left_idx, pivot_idx) + if right_idx > pivot_idx + 1: + quick_sort_helper(nums, pivot_idx + 1, right_idx) + +def quick_sort(nums): + quick_sort_helper(nums, 0, len(nums) - 1) +``` + +### Runtime + +The best case performance of quick sort is O(n log n), but depending on the structure of the list, quick sort's performance can vary. + +If the pivot happens to be the median of the list, then the list will be divided in half after the partition. + +In the worst case, however, the list will be divided into an N - 1 length list and an empty list. Thus, in the worst possible case, quick sort has O(N2) performance, since we'll have to recursively quicksort (N - 1), (N - 2), ... many lists. However, on average and in practice, quick sort is still very fast due to how fast swapping array elements is. + +The space complexity for this version of quick sort os O(log N), due to the number of call stacks created during recursion, but an iterative version can make space complexity O(1). + +**Summary** + +| Worst case | Best case | Stable | In-place| +|:----------:|:---------:|:------:|:-------:| +| O(n2) | O(n log n)| ❌ | ✅ | + +## Insertion sort + +In **insertion sort**, we incrementally build a sorted list from the unsorted list. We take elements from the unsorted list and insert them into the sorted list, making sure to maintain the order. + +This algorithm takes O(n2) worst time, because looping through the unsorted list takes O(n) and finding the proper place to insert can take O(n) time in the worst case. However, if the list is already sorted, insertion sort takes O(n) time, since insertion time will be O(1). Insertion sort can be done in-place, so it takes up O(1) space. + +Insertion sort is easier on linked lists, which have O(1) insertion whereas arrays have O(n) insertion because in an array, inserting an element requires shifting all the elements behind that element. + +**Summary** + +| Worst case | Best case | Stable | In-place| +|:----------:|:---------:|:------:|:-------:| +| O(n2) | O(n)| ✅ | ✅ | + +## Selection sort + +**Selection sort** incrementally builds a sorted list by finding the minimum value in the rest of the list, and swapping it to be in the front. + +It takes O(n2) time in general, because we have to loop through the unsorted list which is O(n) and in each iteration, we search the rest of the list which always takes O(n). Selection sort can be done in-place, so it takes up O(1) space. + +| Worst case | Best case | Stable | In-place| +|:----------:|:---------:|:------:|:-------:| +| O(n2) | O(N2)| ❌ | ✅ | + +## Radix sort + +**Radix sort** is a situational sorting algorithm when you know that the numbers you are sorting are bounded in some way. It operates by grouping numbers in the list by digit, looping through the digits in some order. + +For example, if we had the list ```[100, 10, 1]```, radix sort would put 100 in the group which had 1 in the 100s digit place and would put (10, 1) in a group which had 0 in the 100s digit place. It would then sort by the 10s digit place, and finally the 1s digit place. + +Radix sort thus needs one pass for each digit place it is sorting and takes O(KN) time, where K is the number of passes necessary to cover all digits. + +| Worst case | Best case | Stable | In-place| +|:----------:|:---------:|:------:|:-------:| +| O(kn) | O(kn)| ✅ (if going through digits from right to left) | ❌ | + +## Summary + +|Sort | Worst case | Best case | Stable | In-place| +|:-:||:----------:|:---------:|:------:|:-------:| +|Merge sort | O(n log n) | O(n log n) | ✅ | ❌ | +|Quick sort | O(n2) | O(n log n)| ❌ | ✅ | +| Insertion sort | O(n2) | O(n)| ✅ | ✅ | +|Selection sort | O(n2) | O(N2)| ❌ | ✅ | +|Radix sort| O(kn) | O(kn)| ✅ (if going through digits from right to left) | ❌ | diff --git a/strings_arrays/sorting_colors.md b/strings_arrays/sorting_colors.md new file mode 100644 index 0000000..f47b0c2 --- /dev/null +++ b/strings_arrays/sorting_colors.md @@ -0,0 +1,81 @@ +## Problem +Given N objects colored red, white and blue, sort them *in-place* in red-white-blue order. + +Red, white, and blue objects are represented as 0, 1, and 2 respectively. + +Example: +```python +>>> colors = [2, 1, 2, 0, 0, 1] +>>> sort_colors(colors) +>>> colors +[0, 0, 1, 1, 2, 2] +``` + +## Approach #1: Merge or quick sort +### Approach +The problem is asking us to sort a list of integers, so we can use an algorithm like merge sort or quick sort. + +### Time and space complexity +Both of these sorting algorithms have O(n log n) worst case time complexity and, because we sort in-place, O(1) space complexity. + +## Approach #2: Counting sort +### Approach +We know that the numbers we are sorting are 0, 1, or 2. This means we can sort more efficiently by simply counting the numbers of times each of the three values occurs and modifying the list in-place to match the counts in sorted order. + +### Implementation +```python +from collections import defaultdict +def sort_colors(colors): + counts = defaultdict(int) + for num in colors: + counts[num] += 1 + idx = 0 + while idx < counts[0]: + colors[idx] = 0 + idx += 1 + while idx < counts[0] + counts[1]: + colors[idx] = 1 + idx += 1 + while idx < counts[0] + counts[1] + counts[2]: + colors[idx] = 2 + idx += 1 +``` + +### Time and space complexity +This solution has complexity O(n), since we loop through the list once, then loop through the entire dictionary to modify our list. This solution takes up O(1) space, since everything is done in place and the dictionary has a constant size. + +## Approach #3: Three-way partition +This approach uses multiple pointers. Reading the [two pointer guide](https://guides.codepath.com/compsci/Two-pointer) may be helpful. + +### Approach +Although we cannot asymptotically do better than O(n), since we need to pass through the list at least once, we can limit our code to only making one pass. This will be slightly faster than approach #2. + +We can accomplish this by recognizing that sorting an array with three distinct elements is equivalent to a partition operation. Recall that in quick sort, we partition an array to put all elements with values less than a pivot on the left and elements with values greater than a pivot to a right. Since we only have three potential values in our list, partitioning using the middle value as a pivot will effectively sort the list. + +This particular type of partition is a bit tricky though because we're partitioning on the middle element (the 1's) of our list. It's called a three-way partition, since we are also grouping together elements that are equal in the middle (the 1's). + + +### Implementation +```python +def sort_colors(colors): + left, middle, right = 0, 0, len(colors) - 1 + while middle <= right: + if colors[middle] == 0: + colors[middle], colors[left] = colors[left], colors[middle] + left += 1 + middle += 1 + elif colors[middle] == 1: + middle += 1 + elif colors[middle] == 2: + colors[middle], colors[right] = colors[right], colors[middle] + right -= 1 + middle += 1 +``` + + +### Time and space complexity +This solution has also has time complexity O(n), but only takes one pass since it uses two pointers that stop moving when one moves past the other. + +It is slightly faster than the counting sort and is O(1) space, since it is in-place. + +**Note:** This problem is also known as the [Dutch flag problem](https://en.wikipedia.org/wiki/Dutch_national_flag_problem). diff --git a/strings_arrays/strings_arrays.md b/strings_arrays/strings_arrays.md new file mode 100644 index 0000000..3cf5d07 --- /dev/null +++ b/strings_arrays/strings_arrays.md @@ -0,0 +1,38 @@ +## Arrays +An **array** is a data structure that holds a fixed number of objects. Because arrays have fixed sizes, they are highly efficient for quick lookups regardless of how their size. However, there is a tradeoff with this fast access time: any insertion or deletion from the middle of the array requires moving the rest of the elements to fill in or close the gap. To optimize time efficiency, try to add and delete mostly from the end of the array. + +Arrays commonly come up in interviews, so it's important to review the array library for the language you code in. + +**Tips:** +* Off-by-one errors can happen often with arrays, so be wary of potentially over-indexing as it will throw an error +* Try to add elements to the back of an array instead of the front, as adding to the front requires shifting every element back +* In Java, arrays are a fixed size so consider utilizing an [ArrayList](https://docs.oracle.com/javase/8/docs/api/java/util/ArrayList.html) instead if you need to dynamically alter the size of the array. + +## Strings +**Strings** are a special kind of array – one that only contains characters. They commonly come up in interview questions, so it's important to go through the string library for the language you're most comfortable with. You should know common operations such as: getting the length, getting a substring, splitting a string based on a delimiter, etc. + +It's important to note that whenever you mutate a string, a new copy of the string is created. There are different ways to reduce the space utilized depending on the language: +* In Python, you can represent a string as a list of characters and operate on the list of character instead. +* In Java, you can utilize the [StringBuffer](https://docs.oracle.com/javase/7/docs/api/java/lang/StringBuffer.html) class to mitigate the amount of space utilized if you need to mutate a string. + +## Patterns List +* [Two pointer](https://guides.codepath.com/compsci/Two-pointer) +* [Binary Search](https://guides.codepath.com/compsci/Binary-Search) + +### Strings +#### General guide +* [Coding for Interviews Strings Guide](http://blog.codingforinterviews.com/string-questions/) + +#### Strings in C++ + * [Character Arrays in C++](https://www.youtube.com/watch?v=Bf8a6IC1dE8) + * [Character Arrays in C++ Part 2](https://www.youtube.com/watch?v=vFZTxvUoZSU) + +### Arrays +#### General guide + * [InterviewCake Arrays](https://www.interviewcake.com/concept/java/array) + +#### Python arrays +* [Google developer lists guide](https://developers.google.com/edu/python/lists) +#### Java arrays + * [InterviewCake DynamicArray](https://www.interviewcake.com/concept/java/dynamic-array-amortized-analysis?) + * [ArrayList Succinctly Guide](https://code.tutsplus.com/tutorials/the-array-list--cms-20661) diff --git a/strings_arrays/two_pointer.md b/strings_arrays/two_pointer.md new file mode 100644 index 0000000..5c1db68 --- /dev/null +++ b/strings_arrays/two_pointer.md @@ -0,0 +1,140 @@ +The **two pointer method** is a helpful technique to keep in mind when working with strings and arrays. It's a clever optimization that can help reduce time complexity with no added space complexity (a win-win!) by utilizing extra pointers to avoid repetitive operations. + +This approach is best demonstrated through a walkthrough, as done below. + +## Problem: Minimum size subarray sum + +Given an array of n positive integers and a positive integer s, find the minimal length of a contiguous subarray of which the sum ≥ s. If there isn't one, return 0 instead. + +**Example:** +```python +>>> min_sub_array_length([2,3,1,2,4,3], 7) +2 +``` + +Explanation: the subarray [4,3] has the minimal length under the problem constraint. + +## Solution #1: Brute Force +### Approach + +When first given a problem, if an optimal solution is not immediately clear, it's better to have any solution that works than be stuck. With this problem, a brute force solution would be to generate all possible subarrays and find the length of the shortest subarray that sums up to a sum that is greater than or equal to the given number. + +### Implementation +```python +def min_sub_array_length(nums, sum): + min_length = float("inf") + for start_idx in range(len(nums)): + for end_idx in range(start_idx, len(nums)): + subarray_sum = get_sum(nums, start_idx, end_idx) + if subarray_sum >= sum: + min_length = min(min_length, end_idx - start_idx + 1) + return min_length if min_length != float("inf") else 0 + +def get_sum(nums, start_index, end_index): + result = 0 + for i in range(start_index, end_index + 1): + result += nums[i] + return result +``` + +### Time and space complexity + +The time complexity of this solution would be O(n3). The double for loop results in O(n2) calls to get_sum and each call to get_sum has a worst case run time of O(n), which results in a O(n2 * n) = **O(n3) runtime**. + +The space complexity would be **O(1)** because the solution doesn't create new data structures. + +## Improvements +#### Optimization #1: +**Keep track of a running sum instead of running `get_sum` in each iteration of the inner `end_idx` for loop** + +In the brute solution, a lot of repetitive calculations are done in the inner `end_idx` for loop with the `get_sum` function. Instead of recalculating the sum from elements `start_idx` to `end_idx` in every iteration of the `end_idx` loop, we can store a `subarray_sum` variable to save calculations from previous iterations and simply add to it in each iteration of the `end_idx` loop. + +```python +def min_sub_array_length(nums, sum): + min_length = float("inf") + for start_idx in range(len(nums)): + subarray_sum = 0 + for end_idx in range(start_idx, len(nums)): + subarray_sum += nums[end_idx] + if subarray_sum >= sum: + min_length = min(min_length, end_idx - start_idx + 1) + return min_length if min_length != float("inf") else 0 +``` + +This optimization reduces the time complexity from O(N3) to O(N2) with the addition of a variable to store the accumulating sum. + + +#### Optimization #2: +**Reduce number of calculations by terminating the inner `end_idx` for loop early** + +With the improved solution, we can further reduce the number of iterations in the inner for loop by terminating it early. Once we have a `subarray_sum` that is equal to or greater than the target sum, we can simply move to the next iteration of the outer for loop. This is because the questions asks for minimum length subarray and any further iterations of the inner for loop would only cause an increase in the subarray length. + +```python +def min_sub_array_length(nums, sum): + min_length = float("inf") + for start_idx in range(len(nums)): + subarray_sum = 0 + for end_idx in range(start_idx, len(nums)): + subarray_sum += nums[end_idx] + if subarray_sum >= sum: + min_length = min(min_length, end_idx - start_idx + 1) + continue + return min_length if min_length != float("inf") else 0 +``` + +This is a minor time complexity improvement and this solution will still have a worst case runtime of O(n2). The improvement is nice, but to reduce the runtime from O(n2) to O(n), we would need to somehow eliminate the inner for loop. + + +## Solution #2: Two pointer approach +### Approach: + +The optimal, two pointer approach to this problem utilizing the observations we made in the previous section. The main idea of this approach is that we grow and shrink an interval as we loop through the list, while keeping a running sum that we update as we alter the interval. + +There will be two pointers, one to track the start of the interval and the other to track the end. They will both start at the beginning of the list and move dynamically to the right until they hit the end of the list. + +First, we grow the interval to the right until it exceeds the minimum sum. Once we find that interval, we move the start pointer right as much as we can to shrink the interval until it sums to a number that is smaller than the target sum. + +Then, we move the end pointer to once again to try and hit the sum with new intervals. If growing the interval by moving the end pointer leads to an interval that sums up to at least the target sum, we need to repeat the process of trying to shrink the interval again by moving the start pointer before further moving the end pointer. + +As we utilize these two pointers to determine which intervals to evaluate, we have a variable to keep track of the current sum of the interval as we go along to avoid recalculating it every time one of the pointers moves to the right, and another variable to store the length of the shortest interval that sums up to >= the target sum. + +This push and pull of the end and start pointer will continue until we finish looping through the list. + +### Implementation +```python +def min_sub_array_length(nums, sum): + start_idx = 0 + min_length, subarray_sum = float('inf'), 0 + + for end_idx in range(len(nums)): + subarray_sum += nums[end_idx] + while subarray_sum >= sum: + min_length = min(min_length, end_idx - start_idx + 1) + subarray_sum -= nums[start_idx] + start_idx += 1 + if min_length == float('inf'): + return 0 + return min_length +``` + +### Time and space complexity +The time complexity of this solution is **O(n)** because each element is visited at most twice. In the worst case scenario, all elements will be visited once by the start pointer and another time by the end pointer. + +The space complexity would be **O(1)** because the solution doesn't create new data structures. + +### Walkthrough +Take the example of `min_sub_array_length([2,3,1,2,4,3], 7)`. The left pointer starts at 0 and the right doesn't exist yet. + +As we start looping through the list, our first interval is [2]. We won't fulfill the while loop condition until the list reaches [2, 3, 1, 2] whose sum, 8 is >= 7. We then set the `min_length` to 4. + +Now, we shrink the interval to [3, 1, 2] by increasing `start_idx` by 1. This new interval sums up to less than the target sum, 7 so we need to grow the interval. In the next iteration, we grow the interval to [3, 1, 2, 4], which has a sum of 10 and once again, we satisfy the while loop condition. + +We then shrink the interval to [1, 2, 4]. This is the shortest interval we've come across that sums up to at least the target sum, so we update the `min_length` to 3. + +We now move the `end_idx` pointer and it hits the end of the list, with interval [2, 4, 3]. Then shrink the interval to [4, 3], which sums up to 7, the target sum. This is the shortest interval we've come across that sums up to at least the target sum, so we update the `min_length` to 2. This is the final result that is returned. + +## Takeaways + +This optimization can often be applied to improve solutions that involve the use of multiple for loops, as demonstrated in the example above. If you have an approach that utilizes multiple for loops, analyze the actions performed in those for loops to determine if repetitive calculations can be removed through strategic movements of multiple pointers. + +**Note:** Though this walkthrough demonstrated applying the two pointer approach to an arrays problem, this approach is commonly utilized to solve string problems as well.