diff --git a/README.md b/README.md index 2eb5b28..aae2949 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,45 @@ and is based on the following projects: - [nextjs-fargate-demo](https://github.com/FormidableLabs/nextjs-fargate-demo): We deploy the same Next.js application. - [aws-lambda-serverless-reference][]: The CloudFormation/Terraform infrastructure approach is basically identical to our reference Serverless project. +### Goals + +The main goals of this demo project are as follows: + +1. **Slim down a Next.js Lambda deployment**: The Next.js `target: "serverless"` Node.js outputs are huge. Like really, really big because **each page** contains **all the dependencies**. This project adds a [custom externals handler](./server/util.js) to filter out almost all dependencies in `node_modules` and leave those as normal `require()` calls, thus dramatically decreasing the `pages` bundle sizes. The `node_modules` dependencies are included via `serverless-jetpack` trace mode to keep things tight. + + + If you want to see the difference, we've got an environment variable to skip the Node.js package external excludes, producing default bundles with tons of code per page. Try out the following to see (1) the size of the zip bundle and number of individual files in the zip, and a separate command to see (2) the size of the unzipped index page bundle. + + ```sh + # Slimmer with packages in real node_modules and not bundle. + $ yarn clean && yarn build && yarn lambda:sls package --report + $ du -sh .serverless/blog.zip && zipinfo .serverless/blog.zip | wc -l + 2.1M .serverless/blog.zip + 1241 + $ du -sh .next/serverless/pages/index.js + 52K .next/serverless/pages/index.js + + # Bigger with packages in each page bundle + $ yarn clean && NEXT_SKIP_EXTERNALS=true yarn build && yarn lambda:sls package --report + $ du -sh .serverless/blog.zip && zipinfo .serverless/blog.zip | wc -l + 4.0M .serverless/blog.zip + 293 + $ du -sh .next/serverless/pages/index.js + 2.7M .next/serverless/pages/index.js + ``` + + > ℹ️ **Note**: For a full optimization we'd probably want to see if we could split out application code that is shared across pages as well. For now, we're avoiding a big chunk of `node_modules`, which has a good punch for just 3 actual pages (plus supporting boilerplate). + +2. **Single Lambda/APIGW proxy**: `TODO(ROUTING): INSERT_NOTES` + +### Caveats + +Some caveats: + +1. **Static files**: To make this demo a whole lot easier to develop/deploy, we handle serve static assets _from_ the Lambda. This is not what you should do for a real application. Typically, you'll want to stick those assets in an S3 bucket behind a CDN or something. Look for the `TODO(STATIC)` comments variously throughout this repository to see all the shortcuts you should unwind to then reconfigure for static assets "the right way". +2. **Deployment URL base path**: We have the Next.js blog up at sub-path `/blog`. A consumer app may go instead for root and that would simplify some of the code we have in this repo to make all the dev + prod experience work the same. +3. **Lambda SSR + CDN**: Our React SSR hasn't been tuned at all yet for caching in the CDN like a real world app would want to do. + ## Local development Start with: @@ -25,33 +64,51 @@ Start with: $ yarn install ``` -### Next.js Development server +Then we provide a lot of different ways to develop the server. Here is a table of options with current working status: + +| Command | Status | URL | +| ----------------- | ------ | ---------------------------------------------- | +| `dev` | works | http://127.0.0.1:3000/blog/ | +| | works | http://127.0.0.1:3000/blog/posts/ssg-ssr | +| `start` | works | http://127.0.0.1:4000/blog/ | +| | fails | http://127.0.0.1:4000/blog/posts/ssg-ssr | +| `lambda:localdev` | works | http://127.0.0.1:5000/blog/ | +| | fails | http://127.0.0.1:5000/blog/posts/ssg-ssr | +| _deployed_ | works | https://nextjs-sls-sandbox.formidable.dev/blog/ | +| | fails | https://nextjs-sls-sandbox.formidable.dev/blog/posts/ssg-ssr | + +### Next.js Development server (3000) + +The built-in Next.js dev server, compilation and all. ```sh $ yarn dev ``` -and visit: http://127.0.0.1:3000/ +and visit: http://127.0.0.1:3000/blog/ -### Serverless development server +### Node.js production server (4000) -This uses `serverless-offline` to simulate the application running on Lambda. +We have a Node.js custom `express` server that uses _almost_ all of the Lambda code, which is sometimes an easier development experience that `serverless-offline`. This also could theoretically serve as a real production server on a bare metal or containerized compute instance outside of Lambda. ```sh -$ yarn lambda:localdev +$ yarn clean && yarn build +$ yarn start ``` -and visit: http://127.0.0.1:4000/localdev/blog/ +and visit: http://127.0.0.1:4000/blog/ -### Next.js Production server +### Lambda development server (5000) -This repo _doesn't_ use the prod server, but if you want to create it, here you go: +This uses `serverless-offline` to simulate the application running on Lambda. ```sh -$ yarn build -$ yarn start +$ yarn clean && yarn build +$ yarn lambda:localdev ``` +and visit: http://127.0.0.1:5000/blog/ + ## Deployment We target AWS via a simple command line deploy. For a real world application, you'd want to have this deployment come from your CI/CD pipeline with things like per-PR deployments, etc. However, this demo is just here to validate Next.js running on Lambda, so get yer laptop running and fire away! @@ -172,6 +229,10 @@ We use AWS IAM users with different privileges for these commands. `FIRST.LAST-a is required (to effect the underlying CloudFormation changes). ```sh +# Build for production. +$ yarn clean && yarn build + +# Deploy $ STAGE=sandbox aws-vault exec FIRST.LAST-admin -- \ yarn lambda:deploy @@ -184,9 +245,11 @@ $ STAGE=sandbox aws-vault exec FIRST.LAST-admin -- \ See the [aws-lambda-serverless-reference][] docs for additional Serverless/Lambda (`yarn lambda:*`) tasks you can run. -`yarn lambda:info` gives the current APIGW endpoints. As a useful helper we've separately hooked up a custom domain for `STAGE=sandbox` at: +As a useful helper we've separately hooked up a custom domain for `STAGE=sandbox` at: https://nextjs-sls-sandbox.formidable.dev/blog/ +> ℹ️ **Note**: We set `BASE_PATH` to `/blog` and _not_ `/${STAGE}/blog` like API Gateway does for internal endpoints for our references to other static assets. It's kind of a moot point because frontend assets shouldn't be served via Lambda/APIGW like we do for this demo, but just worth noting that the internal endpoints will have incorrect asset paths. + [aws-lambda-serverless-reference]: https://github.com/FormidableLabs/aws-lambda-serverless-reference [aws-vault]: https://github.com/99designs/aws-vault diff --git a/components/layout.js b/components/layout.js index 7c0f470..a9bf99a 100644 --- a/components/layout.js +++ b/components/layout.js @@ -10,6 +10,7 @@ export default function Layout({ children, home }) { return (
+ {/* TODO: Favicon will need separately asset handling to be at root slot. */} {name} @@ -39,7 +40,7 @@ export default function Layout({ children, home }) { {name} diff --git a/lib/posts.js b/lib/posts.js index 53d8653..28574c2 100644 --- a/lib/posts.js +++ b/lib/posts.js @@ -1,21 +1,25 @@ import fs from 'fs' +import { promisify } from 'util' import path from 'path' import matter from 'gray-matter' import remark from 'remark' import html from 'remark-html' +const readdir = promisify(fs.readdir); +const readFile = promisify(fs.readFile); const postsDirectory = path.join(process.cwd(), 'posts') -export function getSortedPostsData() { +export async function getSortedPostsData() { // Get file names under /posts - const fileNames = fs.readdirSync(postsDirectory) - const allPostsData = fileNames.map(fileName => { + const fileNames = await readdir(postsDirectory) + const allPostsData = await Promise.all(fileNames.map(async fileName => { // Remove ".md" from file name to get id const id = fileName.replace(/\.md$/, '') // Read markdown file as string const fullPath = path.join(postsDirectory, fileName) - const fileContents = fs.readFileSync(fullPath, 'utf8') + // TODO: On not found, wrap ENOENT and push back to a 404. + const fileContents = await readFile(fullPath, 'utf8') // Use gray-matter to parse the post metadata section const matterResult = matter(fileContents) @@ -25,7 +29,7 @@ export function getSortedPostsData() { id, ...matterResult.data } - }) + })) // Sort posts by date return allPostsData.sort((a, b) => { if (a.date < b.date) { @@ -36,8 +40,8 @@ export function getSortedPostsData() { }) } -export function getAllPostIds() { - const fileNames = fs.readdirSync(postsDirectory) +export async function getAllPostIds() { + const fileNames = await readdir(postsDirectory) return fileNames.map(fileName => { return { params: { @@ -49,7 +53,7 @@ export function getAllPostIds() { export async function getPostData(id) { const fullPath = path.join(postsDirectory, `${id}.md`) - const fileContents = fs.readFileSync(fullPath, 'utf8') + const fileContents = await readFile(fullPath, 'utf8') // Use gray-matter to parse the post metadata section const matterResult = matter(fileContents) diff --git a/next.config.js b/next.config.js new file mode 100644 index 0000000..5e22f47 --- /dev/null +++ b/next.config.js @@ -0,0 +1,34 @@ +"use strict"; + +const { nextExternals } = require("./server/util"); + +// **NOTE**: We set a base path that assumes Lambda staging _and_ our +// APIGW proxy base path (of `blog` by default). Many real world apps will +// just have a root base path and it's probably easier than this. +const { BASE_PATH, NEXT_SKIP_EXTERNALS = "false" } = process.env; +if (!BASE_PATH) { + throw new Error("BASE_PATH is required"); +} + +module.exports = { + target: "serverless", + basePath: BASE_PATH, + assetPrefix: BASE_PATH, + env: { + BASE_PATH + }, + webpack: (config, { isServer }) => { + if (isServer) { + // Add more information in the bundle. + config.output.pathinfo = true; + + // TODO: Replace with `webpack-next-externals` when ready. + // // Keep `node_modules` as runtime requires to help slim down page bundles. + // config.externals = (config.externals || []).concat( + // NEXT_SKIP_EXTERNALS === "true" ? [] : nextExternals() + // ); + } + + return config; + } +}; diff --git a/package.json b/package.json index d32f970..8ca134b 100644 --- a/package.json +++ b/package.json @@ -8,10 +8,11 @@ "license": "MIT", "private": true, "scripts": { - "dev": "next dev", - "build": "next build", - "start": "next start", - "env": "echo export STAGE=${STAGE:-localdev}; echo export SERVICE_NAME=nextjs-serverless; echo export AWS_REGION=${AWS_REGION:-us-east-1}; echo export AWS_XRAY_CONTEXT_MISSING=LOG_ERROR", + "dev": "eval $(yarn -s env) && next dev", + "clean": "rm -rf .next", + "build": "eval $(yarn -s env) && next build", + "start": "eval $(yarn -s env) && node server/index.js", + "env": "echo export STAGE=${STAGE:-localdev}; echo export BASE_PATH=${BASE_PATH:-/blog}; echo export SERVICE_NAME=nextjs-serverless; echo export AWS_REGION=${AWS_REGION:-us-east-1}; echo export AWS_XRAY_CONTEXT_MISSING=LOG_ERROR", "cf:_params": "eval $(yarn -s env) && echo --parameters ParameterKey=Stage,ParameterValue=${STAGE} ParameterKey=ServiceName,ParameterValue=${SERVICE_NAME}", "cf:bootstrap:_stack": "eval $(yarn -s env) && echo --region ${AWS_REGION} --stack-name cf-${SERVICE_NAME}-${STAGE}-bootstrap", "cf:bootstrap:_tmpl": "echo --template-body file://aws/bootstrap.yml ", @@ -30,7 +31,7 @@ "tf:service:apply": "yarn run tf:terraform apply $(yarn -s tf:service:_vars)", "tf:service:_delete": "yarn run tf:terraform destroy $(yarn -s tf:service:_vars)", "lambda:sls": "eval $(yarn -s env) && sls -s ${STAGE}", - "lambda:localdev": "yarn run lambda:sls offline start --httpPort ${SERVER_PORT:-4000} --host ${SERVER_HOST:-0.0.0.0}", + "lambda:localdev": "yarn run lambda:sls offline start --httpPort ${SERVER_PORT:-5000} --host ${SERVER_HOST:-0.0.0.0} --noPrependStageInUrl", "lambda:deploy": "yarn run lambda:sls deploy", "lambda:info": "yarn run lambda:sls info", "lambda:logs": "yarn run lambda:sls logs", @@ -40,16 +41,18 @@ }, "dependencies": { "date-fns": "^2.11.1", + "express": "^4.17.1", "gray-matter": "^4.0.2", "next": "^10.0.0", "react": "17.0.1", "react-dom": "17.0.1", "remark": "^12.0.0", - "remark-html": "^12.0.0" + "remark-html": "^12.0.0", + "serverless-http": "^2.7.0" }, "devDependencies": { "serverless": "^2.21.1", - "serverless-jetpack": "^0.10.7", + "serverless-jetpack": "^0.10.8", "serverless-offline": "^6.8.0" } } diff --git a/pages/index.js b/pages/index.js index f09aed3..7c5762c 100644 --- a/pages/index.js +++ b/pages/index.js @@ -5,6 +5,12 @@ import { getSortedPostsData } from '../lib/posts' import Link from 'next/link' import Date from '../components/date' +// TODO: This _shouldn't need BASE_PATH as next/link is supposed to handle it. +const isServer = () => { + return typeof window === "undefined"; +}; +const LINK_BASE = isServer() ? process.env.BASE_PATH : ""; + export default function Home({ allPostsData }) { return ( @@ -23,7 +29,8 @@ export default function Home({ allPostsData }) {