First Post
I have recently graduated, and I decided to write some technical posts, inspired by a post from a database professor that I read.
I personally found that simply reading the paper doesn’t help me to understand and apply results to my research, so I’ll try to summarize and write down some advantages/disadvantages that arise from that paper.
My main interests are parallel/graph algorithms, database theory (with an emphasis on algorithms), and some side research related to game theory ,nearest neighbor problems, and computational biology(mainly computational genomics). I’m currently trying to diverge as much as possible to explore the state-of-the-art methods in the academic field and identify the problems being studied in various conferences. Most of the papers I’ll cover are from database/algorithm papers.
I’ll summarize the current direction of paper reading first.
Database Theory
Problems that interested me were mainly databases that are applied to genomic datasets, and financial datasets, which are mostly read-heavy(in my opinion, mostly true in biology, but it might not be true for financial datasets). Typically, we utilize OLAP databases for these read-intensive datasets, and columnar-based databases are employed for this type of task. I was first interested in how these databases are analyzed, or researched in the academic area, and read through the details. Sadly, as a theorist, most of the research paper was focused on the design choices and experiment results.
Get Taekseung’s stories in your inbox
Join Medium for free to get updates from this writer.
Then I’ve read through the theoretical database venue, where most of the paper focused on sampling, logical framework, data structure, and the complexity of the specific problem. I first discovered that database theory is closely related to logic and complexity, rather than being an algorithmic problem. There are several areas of interest, including data structures for nearest neighbor search problems, data indexing, and hypergraph decomposition, as well as worst-case optimal join and optimal cost search in query optimization. My first search was trying to find every paper that is related to query processing(or optimization), and I’ve narrowed my interests to these problems:
- Framework/theoretical bounds for columnar-based database.
- Hypergraph(or hypertree) decomposition
- MPC model for optimal join
Number one is not often covered in theory, but it is interesting. Maybe introducing the compression rate or I/O accesses to disks in general could help build these theoretical guarantees.
Number two is a topic that is often used in databases. From my understanding, all relational databases can be expressed as a hypergraph, and this could be applied to databases obliviously.
Number three is a topic that I am more familiar with compared to other issues, but I have one suspicion about this: Is the MPC model really useful? Given the well-known discussion that MapReduce is an incorrect approach compared to parallel databases, is this the right way to analyze a distributed problem? I don’t know if there is a better way to interpret this, but MapReduce itself is no longer widely used in industry, which raises questions about its practicality. Maybe reading the FlumeJava paper might resolve my suspicions here.
I’ll end my first post here, as I need to sleep. I’ll continue with parallel graph algorithms next.
TODO: read FLUME JAVA, column-based database vs row-based database.
