Building DevShelf with Prisma: Indexing 1M+ Open-Source Resources
My engineering experience with Prisma ORM, Prisma Postgres, and Prisma Compute while building a high-performance developer resource discovery platform.
Building a search platform that indexes more than 1 million open-source learning resources comes with some interesting engineering challenges.
That’s what I explored while building DevShelf — a discovery platform for developer tutorials, open-source projects, machine learning resources, and system architecture blueprints.
Beyond building the user interface, my main focus was designing an efficient data layer, handling large-scale ingestion, and keeping resource discovery responsive.
For this project, I used Next.js 16, TypeScript, Prisma ORM, Prisma Postgres, and Prisma Compute.
Here’s what I learned along the way.
Architecture
The infrastructure consists of four primary components:
- Next.js 16: Application layer, request handling, and server-side rendering.
- Prisma ORM: Type-safe database queries, schema management, and bulk operations.
- Prisma Postgres: Persistent storage for resource records and metadata.
- Prisma Compute: Hosting and compute environment for the application.
I particularly liked having application hosting and the PostgreSQL database within the same ecosystem.
How Indexing and Search Work
The foundation of DevShelf is a structured resource catalog.
Rather than storing resources as unstructured content, each resource follows a consistent data model containing:
- Title and description
- Resource URL and author
- Category and difficulty level
- Technology tags
- Creation date and GitHub stars
Indexing: I used Prisma’s createMany functionality for bulk resource ingestion, allowing me to populate the database with more than 1 million records efficiently.
Search: Users can discover resources by searching across titles, descriptions, and technology tags.
Filtering and sorting: The discovery interface supports category and difficulty filters, alongside sorting by popularity, newest resources, difficulty, and title.
At a high level, the request flows from the Next.js application through Prisma ORM to PostgreSQL, which provides the matching resource records for the results interface.
Get Anasibnebelal’s stories in your inbox
Join Medium for free to get updates from this writer.
The objective was simple: make a large resource catalog feel fast and easy to navigate.
What Worked Well
1. Efficient Bulk Operations
Prisma’s createMany performed exceptionally well during large-scale ingestion. I handled over 1 million row insertions without noticeable memory spikes or performance degradation.
2. Sub-second Query Performance
Prisma Postgres delivered consistently sub-second search results during my development testing. This was particularly important for maintaining a responsive discovery experience.
3. Streamlined Deployment
Prisma Compute and its CLI tooling simplified infrastructure management and environment isolation. This reduced setup overhead and allowed me to focus more on application development.
How I Evaluated Performance
I used Prisma query logging and PostgreSQL monitoring to understand database behavior.
My evaluation focused on three areas:
- Query latency: Inspected generated SQL queries and their reported durations through Prisma Client query events.
- Bulk insertion: Observed database activity during large-scale
createManyoperations. - Database behavior: Reviewed PostgreSQL query activity and latency to understand how the database responded to indexing and search workloads.
One important distinction: Prisma’s logged query duration measures elapsed time between the client issuing a query and receiving the database response. It is not exclusively PostgreSQL execution time.
These were development-level observations rather than standardized load-testing benchmarks. I am not presenting formal p95/p99 latency or throughput figures.
What Could Be Better
Despite the positive experience, I see three opportunities for improvement:
1. Granular Observability
More detailed visibility into background worker status, indexing progress, processing throughput, and failed operations would make large-scale ingestion easier to monitor.
2. Cold Start Optimization
Reducing initial cold-start latency on Prisma Compute would further benefit search-heavy, latency-sensitive applications.
3. Advanced Documentation
More production-oriented examples covering complex relational schemas, recursive relationships, and large-scale data modeling would be valuable for developers building sophisticated applications.
💡 Final Thoughts
Building DevShelf gave me an opportunity to explore Prisma beyond everyday CRUD operations.
Working with large-scale data ingestion, database queries, and deployment infrastructure helped me understand how important the data layer is when building resource-intensive applications.
My biggest takeaway: A scalable application isn’t just about storing more data. It’s about designing an efficient pipeline to ingest, organize, retrieve, and deliver that data.
Overall, Prisma provided a strong foundation for DevShelf. There are areas I’d love to see improved, but the experience has been valuable from an engineering perspective.
I’m looking forward to continuing to improve DevShelf and exploring more of the Prisma ecosystem.
🔗 Explore DevShelf
Live Project: https://shelf.anasi.me
Official Prisma References
- Prisma ORM Documentation
- Prisma Client — Bulk Operations
- Prisma Client — Query Logging
- Prisma Postgres
- Prisma Compute
#Prisma #PrismaORM #PostgreSQL #NextJS #TypeScript #SoftwareEngineering #WebDevelopment #DevShelf
