Pigsty (HashiCorp for Databases)
Pigsty was a highlight of my time in VC. I sourced it and conducted due diligence for MiraclePlus (formerly Y Combinator China) to invest in it as part of the S22 batch. When I first came across Ruohang’s project translating a book on data that was trending globally on GitHub, I was immediately intrigued by the author’s dedication and service to the open source community. While I was teaching assistant for a database course in college at Carnegie Mellon, I didn’t know enough and reached out for a conversation out of curiosity. The rest is history. I learned by accident that he had another side hustle which he envisioned as the HashiCorp for databases. I ultimately drove the investment, which provided an opportunity for Ruohang to start a company with the idea and go full time.
The experience has a special place in my heart. Jessica Livingston said in Founders at Work: Stories of Startups’ Early Days that “some kind of magic happens in startups, especially at the very beginning, but the only people there to see it are the founders. The best way to understand what happens is to ask them, so that’s what I did.”
That was the inspiration and motivation for me to start this podcast and blog. When I spoke with founders, I always saw sparkles in their eyes and was deeply touched by their perseverance. I felt lucky to have absorbed the ways in which the world could become a better place on their horizons and wanted to share everything I heard in these conversations, with more than family and friends.
I now have the highest respect for entrepreneurs. Their compassion drives a sustained passion to solve problems that matter. It requires a commitment that almost always intertwines with personal life. To them, it is never a burden but a badge of honor and part of their identity. At one point, devotion blends into the vision.
For the perspectives I gained from these interactions and exposures, I’m forever grateful.
Founder
Q: In the beginning, you caught my attention on GitHub’s trending list actually through another non-coding project — your translation of the book Designing Data-Intensive Applications, which received over 10k stars. What motivated you to work on that? Why did you pick this book in particular?
A: Designing Data-Intensive Applications is a classic in tech, like a Bible that captures the entire landscape of the data world with fascinating illustrations. I translated this book out of interest and a desire to learn. Translating is the best way to learn. If you can assimilate the content and phrase it in another language, you have fully grasped the concepts.
I open-sourced it on GitHub because it is a model I appreciate. It took only one round of automatic translation and 8 chapters of manual translation before the project started running. The online community then chimed in with additional input for refinements.
Q: You used to work at Alibaba, Tantan, and Apple and had experience with different tech stacks. Why did you pick database as the theme for your startup?
A: Simple reason — software is eating the world and database stands at the core of information systems. I worked on big data for fintech applications at Alibaba, database administration for internet applications at Tantan, and software development for industrial internet monitoring at Apple. Underneath the seemingly different scenarios and technologies is a common thread — the database PostgreSQL.
I made up my mind to go down the database path during the first year of my career. My first job at Alibaba was writing SQL queries on data warehouses to analyze and process data as a data mining engineer. I took almost all kinds of technical roles that year — software and hardware engineer, algorithm engineer, data analyst, frontend and backend engineer. After these experiences, I became convinced that database indeed lies at the center of information technology, at the intersection of infrastructure and applications. If you dive into the space, you will for sure meet it somewhere down the road.
As to why I picked database for my startup idea, the model I usually employ to decide whether an undertaking is worth pursuing looks at 4 criteria — capability, what I am good at; interest, what I enjoy doing; return, if there will be people writing checks for it; meaningfulness, how much impact it will have on the world. To me, database work checks all of the 4 boxes.
Q: Community participation is critical for open source projects to grow and scale. Does Pigsty already have its own open source community? Does it maintain any relationship with the PostgreSQL (PG) community at large?
A: Pigsty currently has a community of over 300 users. Meanwhile, I am on the open source committee of the official PG Community in China. Initially, Pigsty was a 100% nonprofit open source project whose mission was to promote PostgreSQL. Since users in our PG community often complained about the lack of readily available tools and difficulty in talent hiring in the ecosystem, we came up with the idea of a one-stop solution for PG.
Later, some of our users sought additional business support in the production environment, so we started a company. Pigsty’s relationship with the PG community is similar to the relationship between RedHat and the Linux community. Essentially, we contribute to the community and make a profit in return.
Likes of TiDB had a product first before the community got built. PG already had a large community but was lacking a unifying product. Pigsty filled in the gap by facilitating database use for the entire community.
Q: A lot of open source companies started as side hustles or passion projects. You were also in that state when we first chatted a few months ago. What made you choose to work on Pigsty full time as an entrepreneur? How are your mindset and work style different now compared to when you ran it only during your spare time?
A: The key consideration for going full time was the excess in return over risk. The return for a venture-backed, full-time commitment is the opportunity to build a software with the potential to disrupt the industry’s ecosystem, complemented by public good for society, valuable experience in entrepreneurship, and potentially, financial freedom. The risk is losing a couple of years’ time and hundreds of thousands of dollars. Clearly, the upside has a premium over the downside.
Pressure is both a pro and a con of the full-time commitment for me. It is essentially putting all eggs in one basket. I like challenges, however. Sometimes people need to give themselves a little push. The mentality is the biggest difference between running it full time versus part time. This kind of pressure can unleash enormous power sometimes.
Q: Many open source project owners I have talked to are still quite hesitant about quitting their jobs to start their own companies. You were once in their shoes. What advice do you have for your peers?
A: Hesitation can come from two sources: incomplete information and unclear values. In the case of incomplete information, it means you haven’t figured out what to do, which can be addressed by collecting more information and conducting industry research to strengthen perspectives.
Not knowing what you want is more tricky. I’d recommend trusting your instincts. If you feel that there’s no better way than doing it full time, you should go for it.
Product & Company
Q: What kind of product is Pigsty? What attributes does it have?
A: In simple words, it is a one-stop PostgreSQL-based database solution. Databases are everywhere, but the bar to using them well is high. Years ago, installing Oracle for simple data backup cost hundreds of thousands of dollars. Nonetheless, the bar to using software has been decreasing. Cloud databases are superior (to Oracle) in terms of usability and cost, but Pigsty is an even better alternative.
Pigsty has three main attributes to highlight — high availability, easy use with advanced functionality, and low cost.
First of all, Pigsty is a database distribution that packages almost all extensions you might need in one go. It takes only a click to install and deploy. You can run it in the production environment as well as on your personal computer. You can create as many databases as you want.
Second, Pigsty has two core capabilities — a monitoring system and a high availability cluster. If we think of a database as an engine, Pigsty is the car. High availability refers to the capability of autonomous driving that keeps the car moving over breakdowns of specific components. Similarly, Pigsty is resilient to hardware failures.
Additionally, Pigsty has an unparalleled monitoring system that dissects the real-time status of databases as a BI (business intelligence) system does. Monitoring system is as important as the dashboard in a car. To drive well, you need to have a good dashboard and watch it closely.
Get Techcelerated’s stories in your inbox
Join Medium for free to get updates from this writer.
Last but not the least, cloud databases are cheaper than commercial databases but are still more expensive than open source databases. Pigsty is an open source and free software. If you purchase cloud servers from vendors and deploy them by yourself, you’d still be saving costs by half.
Q: How much can you exactly save for customers?
A: Pigsty can achieve 50% to 80% of cost savings overall. A cloud vendor will charge approximately 160k RMB per year for a cloud database and 80k RMB per year for a cloud server. However, if you purchase a server that you build and deploy yourself, it costs only 30k RMB.
Q: There are a ton of database options in the market. Oracle and MySQL are still the most popular. Who are the major competitors for PG? What are PG’s pros and cons?
A: PG’s strongest competitors are Oracle and MySQL. MySQL is the most popular database among all but is currently in a disadvantaged position. Oracle preceded and acquired MySQL, which is being chased by many NewSQL products like TiDB that are advertising their compatibility with MySQL in an effort to replace it. Additionally, MySQL is far less rigorous than PG in processing and analyzing data while less swift than NoSQL databases like Redis, Elasticsearch, and MongoDB in handling internet companies’ heavy data flows.
Oracle is a longstanding commercial database with loaded functions embraced by state-owned enterprises and banks with deep pockets. With its high price being the biggest drawback, Oracle is the most robust and second most popular database.
PG can go far because it keeps up with both Oracle’s upside of robustness and MySQL’s upside of cost-friendliness. Admittedly, PG has some shortcomings. It covers only 85% of Oracle’s functions. It is relatively less popular than MySQL and hence has a lower talent supply.
Q: With approximately 80% of Oracle’s functions, can PG satisfy the needs of most companies?
A: I think so. Oracle is like Microsoft Office, 80% of whose functions are never used by most people in their lifetime.
Q: As you mentioned, there has been a surge of open source NewSQL products that have gained a lot of traction lately, such as TiDB, which is also compatible with MySQL. Are you concerned they might replace Pigsty as well?
A: Pigsty is not the same kind of products as NewSQL databases — it is a database solution.
NewSQL databases like TiDB, CockroachDB, and YugabyteDB have their own strengths and all contribute to the open source community. I applaud their efforts. The problem here, though, is market adoption. They are an overkill for regular use cases.
The distributed architecture that NewSQL databases employ to achieve horizontal scalability and high availability limits functionality to simple queries. Essentially, they sacrifice quality for quantity, which is not what 99% of companies are solving for. In my experience with users, a single-server database can sufficiently cover the majority of data volume needs.
The ancestor of NewSQL databases is Google Spanner, which was created to provide unlimited scale. Nonetheless, only a handful of internet companies have a few scenarios in which the traffic is close to that of Google and such a scale might come in handy. These internet companies typically don’t like to pay for software, however. Therefore, those who are most likely to benefit from NewSQL are not writing checks.
Q: Gartner predicted that 75% of all databases will be deployed or migrated to a cloud platform by 2022. You don’t seem to worry about cloud databases’ threats to PG.
A: I find Gartner’s prediction too exaggerating to be meaningful. The parameters used for measurement and definition of cloud platforms are vague.
The public cloud model jeopardizes the open source community. Cloud providers make a profit from selling open source software but might not allow their employees to contribute to open source projects accessible to all, which hinders the community’s development.
Cloud databases do pose threats to PG but I’m not concerned. First of all, cloud computing doesn’t solve the fundamental issue of trust, which precedes technological advancement in commercial activities. Cloud providers are not truly neutral parties and there’s no guarantee that companies’ data, which is their lifeblood, won’t be peeped at or stolen. There will always be some users who refuse to be hooked unless the trust issue gets resolved.
Second, even though cloud databases are cheaper than commercial databases, they cost more than deploying free open source software on one’s own machine with the support of an open source cloud monitoring system like Pigsty.
The big picture is that cloud providers are eating open source software while cloud-native and multi-cloud deployment is eating cloud providers.
Market
Q: Why are we seeing that PG’s market share in China is still way below its average share in the world?
A: MySQL is to blame here. Currently, MySQL’s market share to PG’s is 1:1 overseas but 10:1 in China, which is caused by the path dependence of Chinese internet companies. MySQL and PG are both open source relational databases. The key to open source software’s growth is its ecosystem that gets fortified for free by user contributions. The scale of the ecosystem directly translates to the speed of the software’s iterative development. MySQL’s ecosystem is a lot bigger than PG’s in China nowadays. When the internet first emerged in China, MySQL was widely adopted over PG for its usability and hence got to grow with internet companies from the onset. That period was known as LAMP, the time when using Linux, Apache, MySQL, and PHP together could build a website. As MySQL clicked with internet companies’ need for something quick and easy, its market share has outsized PG’s over time.
Q: How can you be so certain that PG can expect a significant jump in popularity in the future?
A: PG has a lot of momentum driven by three forces.
First, in the context of open source software eating commercial software, the market share of open source databases like PG and MySQL has already exceeded that of commercial databases like Oracle and SQL Server.
Second, PG is advanced. With functionality in the ballpark of Oracle’s, PG can satisfy user demands better than MySQL.
Third, under the law of converging to the average, the Chinese market is expected to catch up with the global market in PG adoption. More importantly, PG has become the fastest-growing database on the world stage. An increasing number of new software and applications have opted to use PG as the default database, including GitLab, Jira, Confluence, Discord, and many other SaaS companies that we are all familiar with. Their rising popularity will further raise PG’s.
Q: How far are we from the day when PG becomes mainstream?
A: It already is. In a survey by Stack Overflow last year, PG was voted as the most liked relational database by developers and the second most adopted, with an adoption rate of roughly 40%, preceded only by MySQL’s 50%. The question is not when PG will become mainstream but when it will take the first place. In a historical trend chart published by DB-Engines that captures database popularity, the leading four players are Oracle, MySQL, SQL Server, and PostgreSQL. It is worth noting that the first three are trending down while only PG has followed through an upward trajectory over the last decade. Running a simple linear regression would show that PG can surpass SQL Server by 2024 and MySQL by 2029 to become the most popular database. We are around 7 years away from that day. It’s coming.
Q: I am counting down and cannot wait to revisit your prediction 7 years from now. Hopefully, that day will arrive sooner than later. To your point, when I conducted Pigsty’s due diligence for MiraclePlus, I poked around Hacker News for PG’s defects in people’s comments but saw many praises.
We are seeing a wave of infra tech startups in China. Pigsty is also part of this new force. What opportunities and challenges are you seeing in this market?
A: Software paradigms will continue to evolve. Software is eating the world. Open source is eating software. Cloud providers are eating open source. Cloud-native and multi-cloud deployment are eating cloud providers. I think the biggest opportunity for startups is to take over market shares in software and PaaS from cloud providers in the post-cloud era.
The biggest challenge is competition. That’s just the way it is in China. We have a ton of talent here so people rush to wherever profitable businesses are. Building up competitive advantages is crucial. Of course, overall market shifts would be the single biggest challenge that would overshadow all other factors but we can only focus on the variables that we have control over.
