diff --git a/.github/ISSUE_TEMPLATE/improve-docs-content.md b/.github/ISSUE_TEMPLATE/improve-docs-content.md new file mode 100644 index 0000000000..7f7547643f --- /dev/null +++ b/.github/ISSUE_TEMPLATE/improve-docs-content.md @@ -0,0 +1,17 @@ +--- +name: Improve docs content +about: Suggest improvements or report an error or omission in the docs +title: '' +labels: '' +assignees: '' + +--- + +**Link to the affected page or pages** +Provide URLs of pages on fly.io/docs. + +**Describe what parts of the doc need improvement** +Provide lots of detail about your suggestion for improvement or about the error or omission that you found. If applicable, let us know how you would fix the content. + +**Additional info** +Add any other context about the issue here. If applicable, add screenshots to help explain the suggestion or error. diff --git a/.github/ISSUE_TEMPLATE/improve-the-fly-io-docs-site.md b/.github/ISSUE_TEMPLATE/improve-the-fly-io-docs-site.md new file mode 100644 index 0000000000..a245d25a7b --- /dev/null +++ b/.github/ISSUE_TEMPLATE/improve-the-fly-io-docs-site.md @@ -0,0 +1,17 @@ +--- +name: Improve the fly.io/docs site +about: Suggest improvements or report a site or display error. +title: '' +labels: '' +assignees: '' + +--- + +**Link to the affected page or pages** +If applicable, provide URLs of pages on fly.io/docs. + +**Describe what part of the site needs improvement** +Provide lots of detail about what's going wrong with the navigation or display of the site. + +**Additional info** +Add any other context about the issue here. Add screenshots to help explain the suggestion or error. diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index af691b23be..f8c247f255 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,7 +1,8 @@ ### Summary of changes +### Preview + ### Related Fly.io community and GitHub links -n/a if none ### Notes -n/a if none + diff --git a/.github/workflows/vale-docs-linter.yml b/.github/workflows/vale-docs-linter.yml index 15d62b7290..43404c4cf6 100644 --- a/.github/workflows/vale-docs-linter.yml +++ b/.github/workflows/vale-docs-linter.yml @@ -8,13 +8,14 @@ jobs: runs-on: ubuntu-latest steps: - name: check out repo - uses: actions/checkout@v3 + uses: actions/checkout@v4 with: fetch-depth: 2 - name: get changed files id: changed-files - uses: tj-actions/changed-files@v39 + + uses: tj-actions/changed-files@v46.0.5 with: fetch_depth: 2 diff --git a/.vale.ini b/.vale.ini index b1b4e2cef7..29d5c627a5 100644 --- a/.vale.ini +++ b/.vale.ini @@ -1,9 +1,11 @@ StylesPath = styles -Vocab = Fly-terms +Vocab = fly-terms MinAlertLevel = suggestion +SkippedScopes = script + Packages = proselint, write-good, Google [*{.markerb,md}] @@ -13,7 +15,7 @@ BasedOnStyles = Fly, Google, proselint, Vale, write-good Fly.Exclamation = NO Fly.Headings = NO Fly.Latin = NO -# Fly.Machine = NO # we can do an initial search and replace and then this rule will be less noisy and more useful +Fly.Machine = NO # we can do an initial search and replace and then this rule will be less noisy and more useful Fly.OxfordComma = NO Fly.SentenceLength = NO # turn on later; then reduce further? Fly.Us = NO @@ -76,4 +78,4 @@ Vale.Spelling = NO # replaced by Fly.Spelling # Vale.Terms = NO [formats] -markerb = md # maps our markerb file extension to md for linting \ No newline at end of file +markerb = md # maps our markerb file extension to md for linting diff --git a/about/billing.html.markerb b/about/billing.html.markerb index 9eb560b82e..9dd7689ac7 100644 --- a/about/billing.html.markerb +++ b/about/billing.html.markerb @@ -6,45 +6,128 @@ nav: firecracker redirect_from: /docs/about/credit-cards/ --- -Billing for Fly.io is monthly per organization. Organizations are administrative entities that enable you to add members, share app development environments, and manage billing separately. +## Overview -If you provision services beyond the [free allowances](/docs/about/pricing/#free-allowances), we will charge you actual money at the end of each month. +Billing for Fly.io is monthly per organization. Organizations are administrative entities that enable you to add members, share app development environments, and manage billing. -Refer to our [Pricing page](/docs/about/pricing/) for resource cost details. +New customers and all new organizations (including those created by previously existing customers) are billed monthly for resource usage. Refer to our resource [Pricing page](/docs/about/pricing/) for details. + +--- ## View invoices -You can view and download invoices from the Billing section of your [dashboard](https://fly.io/dashboard/personal/billing) for each organization. +You can view and download invoices from the **Billing** section of your [dashboard](https://fly.io/dashboard/personal/billing) for each organization. -To download a past invoice, click View for the relevant cycle. You'll be sent to the Stripe billing portal where you'll have the option to download the invoice. +To download a past invoice, click **View** for the relevant cycle. You'll be sent to the Stripe billing portal where you'll have the option to download the invoice. -## Volume billing +--- + +### Understand your invoice charges + +The prices of provisioned services are mostly fixed. If you provision Machines and leave them running all month, we’ll charge you a predictable amount for that Machine. + +A common reason for additional charges is extra RAM usage. -Volume billing is pro-rated to the hour and we subtract the free allowances first. For details, see [Volume pricing](/docs/about/pricing/#persistent-storage-volumes). +For a breakdown of charges, check out your invoice in the Billing section of the [dashboard](https://fly.io/dashboard/personal/billing). -If you create a volume, you will be charged for it. You’re billed for volumes that aren’t attached to Machines, and for volumes that are attached to Machines in any state, including stopped Machines. +We’re happy to discuss a refund if you do create compute resources by mistake, or if your app receives unexpected traffic due to an attack and generates a surprisingly large bill. Just send an email to billing@fly.io. -You aren't billed for the 5 days worth of [daily volume snapshots](/docs/apps/volume-manage/#restore-a-volume-from-a-snapshot) that we store for you by default. +--- ## Machine billing -Machines are billed based on the number of CPUs and the RAM used, according to the Machine state and the time they are running (started). +Started Machines are billed per second that they're running (the time they spend in the `started` state), based on the price of a named CPU/RAM combination, plus the price of any additional RAM you specify. + +For example, a Machine described in your dashboard as "shared-1x-cpu@1024MB" is the “shared-cpu-1x” Machine size preset, which comes with 256MB RAM, plus additional RAM (1024MB - 256MB = 768MB). For pricing and available CPU/RAM combinations, read about [Compute pricing](/docs/about/pricing/#compute). + +Stopped and suspended Machines are billed based on their root file system (rootfs) usage per second (the time they spend in the `stopped` or `suspended` state) by $0.15 per GB per month. + +For example, a Machine described in your dashboard as having 1GB of rootfs stopped for 10 days in the entire month will cost $0.05. If you have 3 stopped Machines of 1GB rootfs each stopped for 30 days it will cost $0.45 ($0.15 x 3). + +--- + +## GPU billing + +GPUs are billed per second that the attached Fly Machine is running (the time they spend in the `started` state), based on the per hour cost of the GPU card. Learn more about [pricing for GPUs](/docs/about/pricing/#gpus-and-fly-machines). + +You're also billed for the Fly Machine separately from the GPU. + +--- + +## Volume billing + +Volume billing is pro-rated to the hour and we subtract the free allowances first for the Launch, Scale, and Enterprise plans (all of which are now discontinued for new customers). For details, see [Volume pricing](/docs/about/pricing/#persistent-storage-volumes). + +If you create a volume, you will be charged for it. You're billed for volumes that aren't attached to Machines, and for volumes that are attached to Machines in any state, including stopped Machines. Volumes in `pending_destroy` state do not accrue charges. + +--- + +## Volume snapshot billing + +Volume snapshot billing is pro-rated to the hour and we subtract the free allowance first. For details, see [Volume Snapshots pricing](/docs/about/pricing/#volume-snapshots). + +If you create a snapshot of a volume (either manually or through automatic daily snapshots), you will be charged for the storage space it occupies. + +Snapshots are stored incrementally, so you're only charged for data that has changed since the previous snapshot for the volume. + +For example, if you have a 10GB volume with 2GB of actual data and 5 daily snapshots with minimal changes between them, you might be charged for around 2.5GB of total snapshot storage rather than 50GB (5 × 10GB). -For example, “1x Shared 1024MB” is one CPU and 1024 MB RAM. To estimate costs, multiply the the price of “shared-cpu-1x” by the number of CPUs used and the number of seconds the Machine was running (price x cpu count x seconds). Then add charges for any extra RAM. Note that we subtract the free allowances first and that pricing is tiered based on usage. For details, see [Compute pricing](/docs/about/pricing/#compute). +--- + +## Unified Billing + +Unified billing allows you to consolidate billing for multiple organizations under a single organization. This means you'll receive a single invoice for all organizations under unified billing. This feature is useful for managing billing for multiple organizations, such as when you have multiple teams or projects. + +**Billing Organization**: The organization that is responsible for paying for the resources used by all organizations linked to it. Any organization in good standing with an attached payment method can be a Billing Organization. + +**Linked Organization**: An organization that is linked to another organization for billing purposes. It otherwise functions as a normal organization. + +### How Unified Billing Works + +- You'll receive a single invoice for all organizations under unified billing. +- Billing Organization must have a payment method on file (credit card or Stripe Link). +- Linked Organizations share credits with their Billing Organization. +- Billing Organizations can have a max of 100 Linked Organizations. + +### How to use Unified Billing + +You can either create a new Linked Organization or link an existing one: + +#### New Organization + +A **Linked Organization** can be created from the [new organization page](/organizations/new). When creating a new organization, you can choose an existing Organization to link it to. + +#### Link an existing Organization + +An existing organization can be converted to a **Linked Organization** from the organization's billing page by clicking the "Link billing to another Organization" button. + +- Only admins can link an organization to another organization. +- We recommend saving any invoices you may need for account purposes before linking an organization. + +--- + +## Some things to consider + +- Each organization has its own [private network](/docs/networking/private-networking/). Organizations are isolated from each other, which might +be advantageous from a security standpoint. +- Cross organization networking is still possible with [Flycast](/docs/networking/flycast/) +- Detaching a Linked Organization from its Billing Organization is not possible at this time. Instead we suggest creating a new organization and moving resources there. + +--- ## Payment options -We process payments through Stripe. Paid plans and monthly invoicing require a credit card or credits. +We process payments through Stripe. Monthly invoicing requires a credit card or credits. ### How we use credit cards We require an active, valid credit card on file for most Fly.io accounts to do things like deploying multiple apps and deploying public images. This is primarily a means to prevent abuse and ensure that we can collect payment at the end of the month. -### Preauthorization Requests +### Pre-authorization Requests -We may send a preauthorization request to the issuing bank to verify a bank will authorize charges. Some credit card companies present these as real charges, which may surprise you. +We may send a pre-authorization request to the issuing bank to verify a bank will authorize charges. Some credit card companies present these as real charges, which may surprise you. -A preauth is really just a hold on a credit card; it is not a real charge. We’ll typically preauthorize a small amount (usually less than $5) after signup, then cancel the authorization immediately. Banks may show the preauth for up to 7 days, even though we have not collected any money. +A pre-auth is really just a hold on a credit card; it is not a real charge. We’ll typically pre-authorize a small amount (usually less than $5) after signup, then cancel the authorization immediately. Banks may show the pre-auth for up to 7 days, even though we have not collected any money. ### If you don't have a credit card @@ -54,12 +137,31 @@ While a credit card is the preferred payment method, if you don't have one, then You can't use a prepaid card as a default (or saved) payment method. You can, however, use a prepaid card to add credits to your account. -## Understand charges beyond the free allowances +--- -The prices of provisioned services are mostly fixed. If you provision Machines and leave them running all month, we’ll charge you a predictable amount for that machine. +## Delete your account -A common reason for additional charges is extra RAM usage. +To delete your Fly.io account, go to Account > Settings > [Delete Account](https://fly.io/user/deactivate) in your dashboard. You'll see a list of clean-up tasks that need to be done before you can delete the account, including: -For a breakdown of charges, check out your invoice in the Billing section of the [dashboard](https://fly.io/dashboard/personal/billing). +- Delete or remove yourself from organizations to avoid leaving behind inaccessible orgs. This does not include your personal organization. +- Delete apps (or remove yourself from the associated organizations). +- Remove certificates so that any custom domains are released. +- Remove extensions to ensure that you don't get billed by partners when you can no longer access your account. + +When you've completed the clean-up tasks, click **Delete** to permanently delete your Fly.io account. + +--- + +## Discontinued/legacy plans + +### Legacy Plan billing + +
+Customers who received a $5 free trial usage credit when signing up for the $5/mo Hobby plan will be billed monthly after that credit is used up. +
+ +Legacy plan billing, and any included usage, is pro-rated over the month. For example, if you signed up for a Scale plan on September 15th, then you'd be charged $14.50 (half of the $29/mo plan cost) right away and you'll have $14.50 of usage included up to September 30th. Then you'll be charged $29 at the start of October and have $29/mo of usage included for that billing cycle. + +### Free resource allowances for Legacy Plan users -We’re happy to discuss a refund if you do create compute resources by mistake, or if your app receives unexpected traffic due to an attack and generates a surprisingly large bill. Just send an email to billing@fly.io. \ No newline at end of file +Fly.io has deprecated plans as of October 7, 2024, but is still honoring them for users who purchased them prior to that date. If your plan includes [free allowances](/docs/about/pricing/#legacy-free-allowances), we subtract them off the top of your usage total. Once your usage exceeds the free allowance for any given resource, we start metering usage of that resource for billing purposes. \ No newline at end of file diff --git a/about/cost-management.html.md b/about/cost-management.html.md new file mode 100644 index 0000000000..8b6002655a --- /dev/null +++ b/about/cost-management.html.md @@ -0,0 +1,64 @@ +--- +title: Cost Management on Fly.io +layout: docs +nav: firecracker +author: kcmartin +date: 2025-10-23 +--- + +## Predicting your Fly.io bill and avoiding surprises + +We’ve done our best to make billing on Fly.io something you can reason about. Machines don’t appear out of nowhere. If something is running, it’s because you launched it, or you configured something that did. In general, when we talk about **autoscaling**, we mean starting or stopping machines you’ve already defined. We don’t quietly spin up extras in the background that turn into mystery charges on your bill. The idea is that your bill should always be traceable to something you can see, name, and plausibly explain when someone asks. If it’s not, we’re happy to help you sort it out. + +
+**Need more help understanding your bill?** Reach out to us at [billing@fly.io](mailto:billing@fly.io). +
+ +## A few examples + +So when you’re budgeting, you can look at what you’ve provisioned and do some quick math. Let’s say you’ve deployed three “shared-1x 1GB” machines in `sjc`. That’s $20.37/month, tops. If you let them stop when idle, you might only pay for a few hours of compute—but the worst-case scenario is still capped at that monthly rate. + +Here are a few more grounded examples: + +- A **staging app** running in the `sjc` region with one `shared-1x 256MB` machine that auto-stops when idle might cost less than **$1/month**. If you leave it running full-time, it’s still only **$2.32/month**. +- A **queue worker** (also in `sjc`) that uses metrics-based autoscaling on `shared-1x 512MB` machines might average 3–5 machines running during busy hours. If those stay up for 50% of the month, you’re looking at **$5.72–9.52/month**, worst-case around **$19 month** if they all run 24/7. + +We recommend budgeting for the “always-on” cost. You can get under that number with auto-stop or bursty usage, but don’t depend on it to hit your budget. + +If your estimate seems high, the most predictable way to save money isn’t fiddling with auto-stop/start settings (since any random request might spin a machine up again), but by just…running fewer machines. Or smaller ones. + +
+**Want to play with numbers?** Try the [Fly.io pricing calculator](https://fly.io/calculator) to get a rough sense of what your setup might cost. For the full breakdown, here’s our [pricing page](/docs/about/pricing/). +
+ +## Metrics-based autoscaling can affect costs + +There is one important exception to the rule that Fly.io doesn’t create machines on your behalf. If you use the **metrics-based autoscaler**, for example to scale a background worker based on queue depth, it _can_ create or destroy machines automatically. + +**When can the metrics-based autoscaler create machines?** When you deploy and configure the [**Fly Autoscaler**](/docs/launch/autoscale-by-metric/), and use `FAS_CREATED_MACHINE_COUNT` instead of `FAS_STARTED_MACHINE_COUNT`, you're giving it permission to create and destroy machines on your behalf. It still uses the machine sizes and regions you specify, and anything it spins up counts toward your bill. [Read more in the docs](/docs/launch/autoscale-by-metric/). + +## Other Stuff to Watch For + +A few more things that can quietly run up your bill if you're not paying attention: + +- **Bandwidth costs can add up.** Outbound data transfer is billed at $0.02/GB in North America and Europe, with higher rates in some other regions. Ingress is free. If you're serving media, syncing large datasets, or replicating across regions, bandwidth can quietly become a big line item. [Learn about data transfer pricing.](/docs/about/pricing/#data-transfer-pricing) +- **Volumes don’t stop billing when your machines do.** Persistent volumes are billed hourly, whether or not your machine is running. That 3GB volume you forgot about in `fra`? It's still costing you. [Read about storage pricing here](/docs/about/pricing/#volumes). +- **Volume snapshots will start billing in January 2026.** If you're using automatic volume snapshots as part of your backup strategy, just be aware: starting next year, they’ll be a billable feature. [We’ve announced the change](https://community.fly.io/t/we-are-going-to-start-charging-for-volume-snapshots-from-january-2026/26202) and will share more details well ahead of time. +- **Managed services live outside your apps.** Fly's [Managed Postgres](/docs/mpg/overview/) (MPG), [Upstash Redis](/docs/upstash/redis/), and [Tigris object storage](/docs/tigris/) don’t get deleted when you delete your apps. You’ll find them in your Dashboard. If you're cleaning up, double-check there too. We've seen people get surprised by bills from leftover services and databases they thought were gone. +- **Dedicated public IPv4 addresses aren’t free.** Each http app gets billed $2/month if you assign it a dedicated public IPv4. If you’ve got a bunch of apps or deploy previews per branch, this can sneak up on you. +- **Internal-only apps still count.** Just because a Fly app doesn’t serve web traffic doesn’t mean it’s free. Background workers, queue processors, cron jobs: they all run machines, and those still cost money. + +## Understanding Free Usage and Overages + +- **Free allowances don’t cap your bill.** We may give you free credits and usage allowances, especially when you’re starting out or have a legacy billing plan. But there’s no soft ceiling. If you go over, we’ll bill you. We don't support billing alerts (yet), so budget accordingly. +- **There is no "free account/free tier" on Fly.io.** We do have a Free Trial program, which you can read about [here](/docs/about/free-trial/). +- **Check your dashboard often.** To spot ballooning costs and overages before they become an issue, check the "current month to date bill" item in your dashboard. + +## Related Reading + +Want to go deeper on scaling strategies, autoscaling configuration, or controlling concurrency? These docs walk through the mechanics: + +- [Autoscale Fly Machines](/docs/blueprints/autoscale-machines/) A practical guide to getting basic autoscaling working with Fly Machines. Covers HTTP-based scaling triggers. +- [Metrics-based Autoscaling](/docs/launch/autoscale-by-metric/) This is the one that _can_ create machines automatically, based on queue depth or another metric you specify. Use it wisely. +- [Setting Concurrency Limits](/docs/blueprints/setting-concurrency-limits/) Shows how to keep your app from getting overwhelmed by too many concurrent requests. Especially useful if you’re scaling horizontally. + diff --git a/about/extensions.html.markerb b/about/extensions.html.markerb index 22fdc4bd7a..ea1b5d0c1e 100644 --- a/about/extensions.html.markerb +++ b/about/extensions.html.markerb @@ -9,7 +9,7 @@ Fly.io is a global cloud service for running full stack apps close to users. We We have two main goals: to deliver a slick developer experience, and to build the best possible platform for running full stack apps. We're looking for partners that share this goal, extending Fly.io with services our customers need. -Services such as managed databases, exception handlers, CI runners or log aggregators are great examples. Check out our [managed Upstash for Redis](https://fly.io/docs/reference/redis/) to get an idea of what's possible, and how we integrate services directly into our CLI. +Services such as managed databases, exception handlers, CI runners or log aggregators are great examples. Check out our [managed Upstash for Redis](https://fly.io/docs/upstash/redis/) to get an idea of what's possible, and how we integrate services directly into our CLI. [Check out our documentation](https://fly.io/docs) to see what our platform has to offer. Contact us at [extensions@fly.io](mailto:extensions@fly.io) to discuss your case! @@ -52,4 +52,4 @@ To fulfil our promise of a performant application in any Fly.io region, latency Your Fly.io network is essentially a global, encrypted LAN, with DNS service discovery and load balancing built-in. This greatly simplifies configuration of services that cluster and gossip. Customers access your service via a single private IP address that automatically routes traffic to the nearest provider VM. -We also offer cost-cutting features such as [automatic VM stop/start based on incoming request volume](https://fly.io/docs/apps/autostart-stop/). Contact us at [extensions@fly.io](mailto:extensions@fly.io) to learn more - we're happy to help you get started! +We also offer cost-cutting features such as [automatic VM stop/start based on incoming request volume](/docs/launch/autostop-autostart/). Contact us at [extensions@fly.io](mailto:extensions@fly.io) to learn more - we're happy to help you get started! diff --git a/about/free-trial.html.md b/about/free-trial.html.md new file mode 100644 index 0000000000..2c97fa4cf1 --- /dev/null +++ b/about/free-trial.html.md @@ -0,0 +1,51 @@ +--- +title: Fly.io Free Trial +layout: docs +nav: firecracker +author: kcmartin +date: 2025-10-27 +--- + +**Fly.io runs your apps close to your users. This page explains how our free trial works and what resources you can use before you need to add a payment method.** + +## Free Trial overview + +A free trial on Fly.io includes 2 hours of machine runtime or 7 days of access, whichever comes first. That’s just enough to get a real app up and running before you decide if it’s a fit for you. + +
If you hit any of the limits below before your 7 days are up, **your trial is considered exhausted**, and your apps will stop until you add a payment method.
+ +Here’s what’s **included** during the trial: + +- **2 total VM hours** (Shared across any machines you launch)
**Note: Trial Machines are set to automatically stop after running for 5 minutes** +- **10 machines max** +- **20GB of volume storage** +- Up to **2 vCPUs per machine** +- **4GB memory per machine** + +These are **not included** in the free trial: + +- Dedicated IPv4 addresses +- Access to performance-optimized vCPUs +- GPU machines + +
You can add a credit card from the dashboard at any time during the trial. This lifts the resource limits and keeps your apps running without interruption. **Note: adding a card ends the free trial** and your usage starts counting toward your bill from that point on.
+ +## Checking usage + +You can see what you’ve used and what’s left under **“Trial Status”** in the Fly.io dashboard. Machines usage (including memory) is tracked per second, while Volumes and Machine count are tracked per hour. + +The free trial is meant for kicking the tires. Spin something up, poke around, deploy a real app or two. If Fly.io fits your use case, great! Add a credit card to keep going. If not, no hard feelings. + +## What happens when the free trial ends? + +If you don’t add a payment method by the end of your 7-day trial, or if you use up the included resources, your apps will stop running. You won’t be able to launch new machines, attach volumes, or deploy changes until billing is set up. + +Whether you're testing the waters or gearing up to launch something real, the free trial gives you room to get started. If Fly.io feels like a fit, adding a payment method from the Dashboard is all it takes to keep going. No surprises, no downtime, no pressure. + +--- + +## Related Reading + +- [Pricing Overview](/docs/about/pricing/) +- [How Billing Works](/docs/about/billing/) +- [Fly.io Pricing Calculator](https://fly.io/calculator) diff --git a/about/healthcare.html.markerb b/about/healthcare.html.markerb index b2402b6552..94f4150595 100644 --- a/about/healthcare.html.markerb +++ b/about/healthcare.html.markerb @@ -5,76 +5,127 @@ sitemap: true nav: firecracker --- -Fly.io is a great place to develop and host healthcare applications! You can get started for free and be up and running with a fully secure solution in minutes. +
+ Illustration by Annie Ruygt of Frankie the hot air balloon examining a green box with A written on it +
-We recognize that healthcare apps and data are different beasts and we're here to protect your patients' data (and keep your auditors happy). We're SOC2 (Type 1) audited, we'll sign BAAs, and we're available to answer questions you might have about how our platform meets your compliance needs [(just ask!)](mailto:sales@fly.io). +Fly.io is a great place to develop and host healthcare applications! You can get started for free and be up and running with a fully secure solution in minutes. -Nailing the security for a HIPAA-compliant application can be a big task, and no hosting provider can do it all for you. But Fly.io has a security-first design and a number of features that make HIPAA simpler: +We recognize that healthcare apps and data are different beasts and we're here to protect your patients' data (and keep your auditors happy). We're SOC2 (Type 2) audited, we'll sign BAAs, and we're available to answer questions you might have about how our platform meets your compliance needs [(just ask!)](mailto:sales@fly.io). + +Nailing the security for a HIPAA-compliant application can be a big task, and no hosting provider can do it all for you. But Fly.io has a security-first design and a number of features that make HIPAA simpler: ## Access Control -##### Database Endpoint Security + + +### Database Endpoint Security + Whether you’re running Fly.io Postgres or your own database, there’s no network ACLs required to ensure that only your application can talk to the database server; databases on Fly.io talk to app servers over 6PN and WireGuard, and never to the public Internet. -##### Anti-Spoofing Controls + +### Anti-Spoofing Controls + Attackers that boot up evil Fly.io apps can’t spoof packets to other Fly.io instances; we use both Linux kernel controls, IP routing, and eBPF programs to make sure of that. It’s kind of a 1998 sort of attack to worry about, but in case your auditor cares, we took care of it. ## Audit -##### Centralizing Logging and Metrics -Fly.io collects logs and metrics from your apps, and can send them wherever you need them to go; you can get a single feed of logs from all your instances into an ELK cluster or a Splunk instance, or aim Grafana at our metrics, so you have visibility and an audit trail of what’s going on with your app.  + +### Centralizing Logging and Metrics + +Fly.io collects logs and metrics from your apps, and can send them wherever you need them to go; you can get a single feed of logs from all your instances into an ELK cluster or a Splunk instance, or aim Grafana at our metrics, so you have visibility and an audit trail of what’s going on with your app. + ## Integrity -##### Hardened Hosting -Apps running on Fly.io run inside Firecracker, a Rust-based, memory-safe KVM hypervisor designed at Amazon as the engine for Fargate. At Fly.io, we take container images from our users and transmogrify them into VMs, for full, no-shared-kernel isolation between applications. Firecracker VMs run as userland processes on our hosts, and are further locked down with modern Linux security tools, including cgroups, file and network namespaces, resource limits, and privilege separation.  -##### Kernel Vulnerability Response + +### Hardened Hosting + +Apps running on Fly.io run inside Firecracker, a Rust-based, memory-safe KVM hypervisor designed at Amazon as the engine for Fargate. At Fly.io, we take container images from our users and transmogrify them into VMs, for full, no-shared-kernel isolation between applications. Firecracker VMs run as userland processes on our hosts, and are further locked down with modern Linux security tools, including cgroups, file and network namespaces, resource limits, and privilege separation. + +### Kernel Vulnerability Response Fly.io is responsible for the security both of our host kernels (of course) and of the guest kernels our apps run in; one less thing for you worry about. ## Authentication -##### Multi-Factor Authentication + +### Multi-Factor Authentication Fly.io supports standard multifactor authentication apps, because of course we do. We feel a little miffed you had to ask. -##### Secrets Management + +### Secrets Management + It’s easy to expose secrets to your running apps without leaking them in configurations: we provide a “write-only” secrets management scheme that keeps app secrets encrypted, exposing them only to running instances of your app, using a token-based system that ensures your plaintext secrets never hit machines not running your apps. ## Confidentiality -##### WireGuard Everywhere + +### WireGuard Everywhere + The Fly.io platform is knitted together out of hosts connected via a WireGuard mesh. Everything talks to everything else over WireGuard. WireGuard is a next-generation in-kernel (and userland) VPN designed by vulnerability researchers for simplicity, auditability, and modern cryptography. The Linux kernel implementation of WireGuard runs in steady state without requiring dynamic memory allocation! WireGuard is great, and is the gold standard for secure network transports. -We run a full mesh of WireGuard. That means that once a request arrives at our edge from the Internet, when we proxy it to a host running your app, that communication occurs over a WireGuard virtual network.   +We run a full mesh of WireGuard. That means that once a request arrives at our edge from the Internet, when we proxy it to a host running your app, that communication occurs over a WireGuard virtual network. + We also use WireGuard in our API. Need to SSH to an instance of your app? That’ll happen over WireGuard. Need to open a Postgres shell? WireGuard. Deploy a new instance using a remote builder running in Fly.io? You guessed it: the Docker protocol stuff is running over WireGuard, from your machine to our hosts. -##### Encryption In Transit -See above! WireGuard runs 256-bit ChaCha20-Poly1305 with an authenticated Curve25519 key exchange.  -##### TLS Everywhere -Fly.io terminates TLS for our users at our edge. We run a fleet of memory-safe, Rust-based proxies that use the Hyper and Rustls libraries to implement HTTPS, in a tight configuration that scores an A grade from Qualys SSL Labs, without you needing to lift a finger.  + +### Encryption In Transit + +See above! WireGuard runs 256-bit ChaCha20-Poly1305 with an authenticated Curve25519 key exchange. + +### TLS Everywhere + +Fly.io terminates TLS for our users at our edge. We run a fleet of memory-safe, Rust-based proxies that use the Hyper and Rustls libraries to implement HTTPS, in a tight configuration that scores an A grade from Qualys SSL Labs, without you needing to lift a finger. + You can terminate TLS in your application instead of at our edge, if you really want to. But you won’t want to. -##### Automatic Certificate Management with LetsEncrypt -Fly.io manages the ACME protocol to securely provision LetsEncrypt TLS certificates, so that’s another thing you’re just not going to have to think much about.  -##### HTTP Strict Transport Security and HTTPS-Only -It’s easy (a single configuration line) to lock your apps to HTTPS-only, using the HSTS protocol to direct browsers exclusively to the secure endpoint of your application and defeating SSL-stripping attacks.  -##### Default-Deny Public Networking -Apps running on Fly.io get routable IPv4 addresses. We’re just givin’ ‘em away! But when your users hit those addresses, they’re not bouncing directly off your app instances; instead, they’re routed to our edge, where we use our memory-safe Rust proxy to direct traffic. What that means for you is that nothing on your app is exposed unless you ask us to expose it. No security group rules or network ACLs required! You’re locked down by default. -##### 6PN Private Networking + +### Automatic Certificate Management with LetsEncrypt + +Fly.io manages the ACME protocol to securely provision LetsEncrypt TLS certificates, so that’s another thing you’re just not going to have to think much about. + +### HTTP Strict Transport Security and HTTPS-Only + +It’s easy (a single configuration line) to lock your apps to HTTPS-only, using the HSTS protocol to direct browsers exclusively to the secure endpoint of your application and defeating SSL-stripping attacks. + +### Default-Deny Public Networking + +Apps running on Fly.io get routable IPv6 addresses and shared IPv4 addresses. But when your users hit those addresses, they’re not bouncing directly off your app instances; instead, they’re routed to our edge, where we use our memory-safe Rust proxy to direct traffic. What that means for you is that nothing on your app is exposed unless you ask us to expose it. No security group rules or network ACLs required! You’re locked down by default. + +### 6PN Private Networking + Modern applications are often composed of ensembles of services. Some of those services are fit for talking to random users on the Internet. Others are best kept under wraps. Fly.io makes it easy to deploy complicated applications built out of multiple services: all Fly.io apps live in a private IPv6 network exclusive to your organization. We use eBPF in the Linux kernel to ensure that private networks can’t talk to each other; they’re completely private, without any extra configuration. We call this feature 6PN (for “IPv6 Private Network”), and it means there’s zero security lockdown work required to deploy a database, Redis cache, or background job scheduler. Your app server will be able to talk to them right away. Nobody outside your organization will be able to talk to them at all. -##### Network Segregation -“Network Segregation” is the term an enterprise IT administrator at a big bank would describe 6PN. Yup, your networks are segregated.  -##### Secure Network Architecture -Sure, you could call 6PN that, too. Did we mention there are no security group rules to review? There are no security group rules to review.  -##### Encryption At Rest + +### Network Segregation + +“Network Segregation” is the term an enterprise IT administrator at a big bank would describe 6PN. Yup, your networks are segregated. + +### Secure Network Architecture + +Sure, you could call 6PN that, too. Did we mention there are no security group rules to review? There are no security group rules to review. + +### Encryption At Rest + Databases like Fly.io Postgres are built on Fly.io Volumes, our persistent storage feature. It works like a drive plugged in and mounted in your app instance. And those drives are block-level encrypted with AES-XTS. We manage the keys for the drives for you, using a token-based orchestration system that ensures the keys are only accessible from privileged processes on hosts actually running your app. Check off another HITRUST CSF requirement from your list! ## Availability -##### High Availability + +### High Availability + Scaling apps across geographic regions is, like, the whole point of the service? I think it’s the whole point? At any rate: it’s definitely a thing we do. -##### DDoS Protection -Fly.io isn’t a DDoS Protection provider. But our upstreams have sophisticated DDoS tools, including blackhole routing and traffic scrubbing, which get regular workouts.  -##### Rolling Deployments + +### DDoS Protection + +Fly.io isn’t a DDoS Protection provider. But our upstreams have sophisticated DDoS tools, including blackhole routing and traffic scrubbing, which get regular workouts. + +### Rolling Deployments + Confidently deploy your Fly.io apps without worrying that you’re going to break everything: we’ll do rolling deploys, with canaries and health-checks, so a known-good version of your app is always running. -##### Advanced App Instance Recovery Space Modulator + +### Advanced App Instance Recovery Space Modulator + If your app crashes or exits, we’ll relaunch the VM. We have the technology. -##### Off-Site Backups + +### Off-Site Backups + Anything stored persistently in a Fly.io volume is backed up on a regular schedule. ## More info -You can check our [community](https://community.fly.io) or [our documentation](https://fly.io/docs/) for answers to your questions. If you can't find what you're looking for want to know more, [get in touch](mailto:sales@fly.io)! - +Our blueprint for [Going to production with healthcare apps](/docs/blueprints/going-to-production-with-healthcare-apps/) runs a developer or operations engineer through the process of evaluating Fly.io’s security for HIPAA healthcare apps, launching a pilot application, signing a BAA, and deploying to production. +You can also check our [community](https://community.fly.io) for answers to your questions. +If you can't find what you're looking for or want to know more, [get in touch](mailto:sales@fly.io). diff --git a/about/index.html.markerb b/about/index.html.markerb index 75da96df75..e3d8d44840 100644 --- a/about/index.html.markerb +++ b/about/index.html.markerb @@ -1,18 +1,37 @@ --- -title: "About Fly" +title: "About Fly.io" layout: docs sitemap: true toc: false nav: firecracker --- -This section is all about the things you also need to know about Fly as a user of the service: +
+ Illustration by Annie Ruygt of Frankie the hot air balloon holding an umbrella to protect a bird from the rain +
-* [_Pricing_](/docs/about/pricing/) : All about how the Fly service is priced. -* [_Healthcare_](/docs/about/healthcare/) : Controls relevant to running healthcare apps on Fly. -* [_Support_](/docs/about/support) : Who to contact and where to go when you need Fly support. -* [_Security_](/docs/about/security) : All about how we handle security and security issues at Fly. -* [_Open Source_](/docs/about/open-source/) : How we help those OSS projects that help us. -* [_Using Our Brand_](/docs/about/brand/) : Assets and guidelines to help represent our brand. -* [_Privacy Policy_](/legal/privacy-policy/) : Fly's policy on privacy, detailed here. -* [_Terms of Service_](/legal/terms-of-service) : The legal terms of service for Fly users. +More details about using and working with the Fly.io platform. + +* **[Pricing](/docs/about/pricing/):** Pricing for compute, storage, and all the Fly.io platform services. + +* **[Billing](/docs/about/billing/):** Learn how billing works on Fly.io. + +* **[Cost Management](/docs/about/cost-management/):** Read this guide to learn how Fly.io billing really works, how to estimate your costs, and what to watch out for, complete with real examples and gotchas. + +* **[Free Trial](/docs/about/free-trial/):** Find out about Fly.io's Free Trial. + +* **[Support](/docs/about/support):** Who to contact and where to go when you need Fly.io support. + +* **[Healthcare](/docs/about/healthcare/):** Controls relevant to running healthcare apps on Fly.io. + +* **[Engineering Jobs and Hiring](/docs/about/extensions/):** Learm more about our hiring process. + +* **[Open Source](/docs/about/open-source/):** How we help those OSS projects that help us. + +* **[Open Source](/docs/about/open-source/):** How we help those OSS projects that help us. + +* **[Using Our Brand](/docs/about/brand/):** Assets and guidelines to help represent our brand. + +* **[Privacy Policy](/legal/privacy-policy/):** Our policy on privacy, detailed here. + +* **[Terms of Service](/legal/terms-of-service):** The legal terms of service for Fly.io users. diff --git a/about/merch.html.markerb b/about/merch.html.markerb new file mode 100644 index 0000000000..858effd3c3 --- /dev/null +++ b/about/merch.html.markerb @@ -0,0 +1,28 @@ +--- +title: Fly.io Merchandise +layout: docs +nav: firecracker +--- + +
+ Fly.io Frankie Merchandise +
+ +## Fly.io Merchandise + +Looking for some superfly swag? We have some Fly merch available in our Redbubble shop. Enjoy! + +Visit our [Redbubble shop](https://www.redbubble.com/people/annieruygt/shop) to check out our latest merchandise. + +## Available Items + +Our Redbubble shop features a variety of Fly.io branded items including: +- T-shirts +- Hoodies +- Stickers +- Mugs +- And more! + +## Contact + +For any questions about our merchandise, please contact us at [annie@fly.io](mailto:annie@fly.io). diff --git a/about/open-source.html.markerb b/about/open-source.html.markerb index fce19b17ac..009099b036 100644 --- a/about/open-source.html.markerb +++ b/about/open-source.html.markerb @@ -2,6 +2,8 @@ title: Fly.io and Open Source layout: docs nav: firecracker +redirect_from: /open-source/ + ---
diff --git a/about/pricing.html.markerb b/about/pricing.html.markerb index 14b0c5ec38..f243365603 100644 --- a/about/pricing.html.markerb +++ b/about/pricing.html.markerb @@ -5,56 +5,111 @@ sitemap: true nav: firecracker --- -A landscape painting with hot air balloons flying over a mountain range. +
+ Illustration by Annie Ruygt of Frankie the hot air balloon demonstrating a pricing chart on a whiteboard +
-## _How it works_ +## How it works -Our pricing is designed to let you launch a small application for free, and scale costs affordably as your needs grow. +Fly.io services are billed per organization, with [Linked Organizations](/docs/about/billing/#unified-billing) reporting resource usage to their parent Billing Organization. Plans get complicated, so we just charge based on usage. Pick and choose which pieces you need for your application; that's all you'll see on your invoice. -Fly.io services are billed per organization. Organizations are administrative entities on Fly.io that enable you to add members, share app development environments, and manage billing separately. Billing is based on the resources provisioned for your apps, pro-rated for the time they are provisioned. Learn more about [billing](/docs/about/billing/). +Organizations are administrative entities on Fly.io that let you add members, share app development environments, and manage billing. [Billing](/docs/about/billing/) is based on the resources provisioned for your apps, pro-rated for the time they are provisioned. -## _Plans_ +Organizations may be subject to automated scaling limits to prevent abuse or to help with capacity planning. Email the address in the error message if you run into such a limit and it's getting in your way. -All plans require a [credit card](/docs/about/billing/#payment-options). Your first organization starts on a free Hobby plan (subject to change), and any subsequent new organizations you create are on the $5/month Hobby Plan. Hobby plans are a straightforward pay-as-you-go option. +All organizations (except for Linked Organizations) require a [credit card](/docs/about/billing/#payment-options) on file. -We don't offer a "free tier." Instead, we offer some free resource allowances that apply to all plans, including the Hobby plan, and includes enough usage per month to run a small full-stack app, full-time, for free.  +## Compute -If you want to scale beyond the included free resources, you can pay for just what you need at the usage-based pricing listed below. +We charge for started and stopped Machines differently. Attached GPUs are charged separately. For more details about how costs are calculated, see [Machine billing](/docs/about/billing/#machine-billing). To understand the difference between `performance` and `shared` CPU types in Machines, see [CPU performance](/docs/machines/cpu-performance). -
-Your organization might be limited to prevent abuse or help with capacity planning. -
+### Started Fly Machines -If you need more support or compliance options, you can upgrade to a [Launch, Scale, or Enterprise plan](https://fly.io/plans). These come with usage included and additional support options. +

+ + +

-For details and to select a different plan, see [Plan Pricing](https://fly.io/plans). + -* Up to 3 shared-cpu-1x 256mb VMs -* 3GB persistent volume storage (total) -* 160GB outbound data transfer +The price of a running [Fly Machine](/docs/machines/) VM is the price of a named CPU/RAM preset, plus about $5 per 30 days per GB of additional RAM. -Additional resources are billed at the usage-based pricing detailed below. +Here's the pricing for named presets and a few standard additional RAM configurations: -## _Compute_ +<%= partial("shared/cpu_mem_machines_pricing") %> -### Fly Machines + -<%= partial("shared/cpu_mem_machines_pricing") %> +### Stopped Fly Machines + +For stopped Machines we charge only for the root file system (rootfs) needed for each Machine. Each 1GB of rootfs for a Machine stopped for 30 days is $0.15. The amount of rootfs needed is defined by your OCI image generated on your app plus a few [containerd](https://containerd.io/+external) tweaks on the underlying file system. + +### Machine reservation blocks + +You'll get a 40% discount when you reserve a block of compute time, either for `performance` Machines or `shared` Machines, in a specific region. +Reservations apply to any number of Machines of the specified CPU class, in the specified region, in any of your organisation's apps. + +The available reservation sizes are: + +**Performance Machines** + +* $144/year for $20/month of usage +* $1,440/year for $200/month of usage +* $14,400/year for $2,000/month of usage + +**Shared Machines** + +* $36/year for $5/month of usage +* $360/year for $50/month of usage +* $3,600/year for $500/month of usage + +You pay the "per year" amount upfront, and each month receive a credit worth the "per month" amount. The credit does not rollover; it's only valid for the month in which it's granted. The credit applies only to CPU and additional RAM charges. + +There's no limit on the number or combinations of blocks that can be purchased. Reservations are backdated to the first day of the month in which they're purchased. -For more details about how costs are calculated, see [Machine billing](/docs/about/billing/#machine-billing). +For example, if you purchase a $36/year `shared` Machines block in `cdg`, you'll pay $36 upfront and receive $5/month of credits applicable to `shared` Machines in `cdg` for 12 months, starting with the month of the purchase. Amortised over 12 months, the $36 upfront cost is $3/month, which is a 40% discount on the $5/month of credits you receive. -### GPUs and Fly Machines +You can set up reservations via self-service in the billing section of your Fly.io [dashboard](https://fly.io/dashboard). They apply to usage starting on the 1st, so setting up reservations any time in the month will give you the credits the entire month. -Pricing for a GPU-enabled Fly Machine is the price of a standard Fly Machine (see [Fly Machines pricing](#fly-machines)) plus the price of the attached GPU. GPU access requires a [Launch, Scale, or Enterprise plan](https://fly.io/plans). +### GPU-enabled Fly Machines + +Pricing for a GPU-enabled Fly Machine is the price of a standard Fly Machine (see above) plus the price of the attached GPU. Like Machines, GPUs are billed by the second when the attached Machine is running. On-demand GPU pricing: +* A10: $1.50/hr per GPU +* L40S: $1.25/hr per GPU * A100 40G PCIe: $2.50/hr per GPU * A100 80G SXM: $3.50/hr per GPU @@ -67,55 +122,167 @@ Reserved and dedicated options: * Discounted rates for reserved GPU Machines and dedicated hosts. -## _Persistent Storage Volumes_ +## Managed Postgres + +The price of running Fly.io Managed Postgres depends on your selected Managed Postgres Plan and the amount of storage your Postgres cluster has. + +Current pricing for Managed Postgres plans and storage is available [here](/docs/mpg#pricing). + +## Persistent Storage Volumes + +### Volumes [Fly Volumes](/docs/volumes/) are local persistent storage for Machines. -* Free: 3GB of total provisioned capacity per organization * $0.15/GB per month of provisioned capacity [Volume billing](/docs/about/billing/#volume-billing) is pro-rated to the hour. You'll be charged for volumes that you create, whether they are attached to a Machine or not, including when an attached Machine is stopped. -## _Network prices_ +### Volume Snapshots + +<% if Time.now.utc < '2026-03-01T00:00:00Z' %> +
+ **New charges** + +

+ Starting January 1st 2026, we're introducing charges for [volume snapshot](/docs/volumes/snapshots/) storage. You'll see the first charges on the invoice issued at the start of February 2026. +

+ +

+ If you're an existing customer, you can check your usage in the **Billing** section of the [dashboard](https://fly.io/dashboard/personal/billing) on your Upcoming Invoice and in the Cost Explorer. +

+
+<% end %> + +* $0.08/GB per month +* First 10GB free each month + +[Volume Snapshot billing](/docs/about/billing/#volume-snapshot-billing) is pro-rated to the hour. + +Automatic daily snapshots with 5 days retention are enabled by default on new volumes. This can be [adjusted](/docs/volumes/snapshots/#set-or-change-the-snapshot-retention-period) or [disabled](/docs/volumes/snapshots/#disable-automatic-daily-snapshots). + +Usage is calculated based on the total stored size of the snapshots, not the provisioned volume size. You're only charged for the actual data stored - if you've written 1GB to a 10GB volume, you'll be charged for around 1GB of snapshot storage. + +Snapshots for each volume are stored incrementally, so you'll only be charged for data that has changed since the previously stored snapshot. + +## Network prices ### Anycast IP addresses -Each application receives a [shared IPv4 address](/docs/reference/services/#shared-ipv4) and unlimited [Anycast IPv6](/docs/reference/services/#ipv6) addresses for global load balancing. +Each application receives a [shared IPv4 address](/docs/networking/services/#shared-ipv4) and unlimited [Anycast IPv6](/docs/networking/services/#ipv6) addresses for global load balancing. Dedicated IPv4 addresses are $2/mo. ### Managed SSL certificates -We use Lets Encrypt to issue certificates, and donate half of our SSL fees to them at the end of each calendar year. +We use Let's Encrypt to issue certificates, and donate half of our SSL fees to them at the end of each calendar year. -* Single hostname certificates: - * Free for the first 10 - * $0.10/mo for additional certificates +* Single hostname certificates: $0.10/mo * Wildcard certificates: $1/mo -### Outbound data transfer +### Data transfer pricing + +We bill for data leaving your app destined for the public internet or for apps or Machines in other regions, including: + +* Data egress to the Internet, from Machine to edge server to Internet +* Data transfer over private network between regions, from Machine to edge server and edge server to Machine +* Data transfer to some extensions like Upstash Redis -We bill for outbound data transfer from the region a VM is running in, inbound transfer is free. +The following types of traffic are free: -<%= partial("shared/network_pricing") %> +* All inbound data transfer +* Data transfer between apps or Machines in the same region (for organizations using granular data transfer rates) +* Data transfer from apps without an assigned IP address (for organizations not using granular data transfer rates) + +Fly.io pricing is per region group for outbound data transfer. You'll see a more detailed breakdown of cost per region and per traffic type on your monthly invoice. + +
+**Important:** Organizations created after July 18 2024 are automatically opted-in to use the granular data transfer rates and are billed at a different rate for private network data transfer between regions, per the following table. Organizations not using granular data transfer rates are billed for all data transfer (excluding that listed as free above) at the "Egress to public internet" rate. +
+ +| Region groups | Egress to public internet cost | Private network cross-region transfer cost | +| --- | --- | --- | +| - North America
- Europe | $0.02 per GB | $0.006 per GB | +| - Asia Pacific
- Oceania
- South America | $0.04 per GB | $0.015 per GB | +| - Africa
- India | $0.12 per GB | $0.050 per GB | + +To opt-in to granular bandwidth pricing, go to the [**Organizations** page](https://fly.io/organizations) in the dashboard, click the organization name to change, then click **Switch to granular bandwidth pricing**. You won't be able to return to using the non-granular data transfer rates once you opt in. + +### Static Egress IPs for Machines + +Static egress IPs for Machines provide dedicated outbound IP addresses for your Machines. When you allocate a static egress IP, you'll get both an IPv4 and IPv6 address for this single price. + +* $0.005 per hour (~$3.60/month) +* Machines do not have a static IP by default + +## Support + +[Community support](https://community.fly.io/) is included for all customers, regardless of usage level. +You can get access to a support plan by purchasing a Standard ($29/month), Premium ($199/month), or Enterprise (starting at $2500/month) package in the **Support** section of your dashboard. For more about Support, see [Support at Fly.io](/docs/about/support/). + +## Fly Kubernetes + +[Fly Kubernetes](/docs/kubernetes/) (FKS) is a managed Kubernetes service that runs on Fly.io. + +* $75/month per cluster +* Plus the cost of [compute](#compute) and [Fly volumes](#persistent-storage-volumes) that you create + +## Extensions + +Fly.io offers managed services operated by third parties, such as [Tigris Object Storage](/docs/tigris) and [Upstash Redis](/docs/upstash/redis/). + +When you provision their services, you become their customer, and you pay their list prices via your monthly Fly.io bill. Charges are updated daily in your Fly.io dashboard. + +You will not be billed separately for: + +* Machines running the services, which are hosted in the provider's account +* IP addresses associated with the service +* Bandwidth to Tigris Object Storage + +You **will** be billed separately for data transfer to these external third-party services. See our [data transfer pricing](#data-transfer-pricing) for details. + +## Unsupported Products + +### Unmanaged Fly Postgres (Unsupported) + +[Fly Postgres](/docs/postgres/) is a PostgreSQL database that you create and then manage yourself. Fly Postgres clusters are Fly Apps that consist of Machines, volumes, and any configured extra memory. + +The [Machine price](#compute) and [volume price](#persistent-storage-volumes) for Fly Postgres are the same as any other Machine and volume you'd run on Fly.io. Assuming the Machines are running all the time, the cost for the preset configurations is about $2/month for a single node cluster for dev projects and from about $82 to $164/month for a three-node production cluster. You don't need to keep the preset configurations, you can [scale your Fly Postgres Machines](/docs/postgres/managing/scaling/) to suit your workload at any time. + +## Discontinued Plans + +Fly.io no longer offers plans to new customers. If you purchased a Launch or Scale plan before October 7, 2024, you can remain on those plans unless you convert to Pay As You Go, delete your payment method, or otherwise stop using Fly.io. + +### Legacy Free allowances + +The following resources were included for free on the Hobby (deprecated), Launch, and Scale plans, and are still honored for any organizations that were on these plans before we sunset them: + +* Up to 3 shared-cpu-1x 256mb VMs +* 3GB persistent volume storage (total) +Outbound data transfer: +* 100 GB North America & Europe +* 30 GB Asia Pacific, Oceania & South America +* 30 GB Africa & India -## LiteFS Cloud +### Paid Hobby plan with Free Trial credit -[LiteFS Cloud](/docs/litefs/cloud-backups) is a service that provides streaming -backups and point-in-time restore for your SQLite-based applications that use -[LiteFS](/docs/litefs). +If you signed up for the now deprecated $5/month paid Hobby plan prior the release of the Pay As You Go plan, you got a one-time $5 free trial credit to let you test-drive Fly.io at no cost. When the free trial credit was used up, we automatically placed your organization on the $5/month Hobby plan, which included $5/month of usage and free allowances. There are no free allowances during the free trial. We'll send you an email when your free trial credit is used up and you won't be charged before that. The free trial credit doesn't expire and applies only to the default (“personal”) organization that we created for you on sign-up. -- $5 per month for up to 10GB of database storage. -- Additional $0.50/GB per month for database storage above 10GB. +To change your plan to the Pay As You Go plan, go to the [**Organizations** page](https://fly.io/organizations) in the dashboard, click the organization name to change, then click **Choose Pay As You Go**. If you change your plan, you won't be able to return to the paid Hobby Plan. +### Legacy (Free) Hobby plan -## _Support_ +If you were on the free Hobby plan at the time that the paid Hobby plan became the default for new organizations, your plan is now called the Legacy Hobby plan. Your costs stay the same as they were, with no monthly subscription fee, and no included usage beyond the free resource allowances. -[Community support](https://community.fly.io) is included for all customers, regardless of usage level. +If you change your plan, you won't be able to return to the Legacy Hobby Plan. -Email support is included with our [Launch, Scale, and Enterprise plans](https://fly.io/plans). +## **Related reading** -For more about Support, see [Support at Fly.io](https://fly.io/docs/about/support/). +- [Billing for Fly.io](/docs/about/billing/) How invoicing, payment methods, and usage tracking work. +- [Cost Management](/docs/about/cost-management/) Best practices for estimating, monitoring, and controlling your spend. +- [Free Trial](/docs/about/free-trial/) What the Fly.io Free trial gives you and when billing begins. +- [Organization Roles & Permissions](/docs/security/org-roles-permissions/) How org structure, permissions and billing interplay. +- [Optimize Compute Costs: Fine‑tune your app](/docs/apps/fine-tune-apps/) Tuning machine size, memory/CPU, and stop‑/start behavior to reduce waste. \ No newline at end of file diff --git a/about/security.html.md b/about/security.html.md deleted file mode 100644 index 48b595447a..0000000000 --- a/about/security.html.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -title: Security at Fly.io -layout: docs -nav: firecracker ---- - -Securing a major hosting platform like [Fly.io](http://Fly.io) is a hard problem, and we take it seriously. - -
-**Reporting issues**: If you have a security concern, or believe you’ve found a vulnerability in any part of our infrastructure, please contact us. You can reach us at [**security@fly.io**](mailto:security@fly.io), and we can provide you with a Signal number if needed to convey sensitive information. -
- -## Our Security Practice - -**Corporate Security (“CorpSec”)** - -CorpSec is the practice of making sure [Fly.io](http://Fly.io) team members have secure access to [Fly.io](http://Fly.io) company infrastructure, and that secured channels are the only exposed channels to [Fly.io](http://Fly.io). CorpSec controls are the primary concern of standards like SOC2. - -- Access to our services and applications is gated on a SSO Identity Provider. -- We require strong, phishing-resistant 2FA in all enrolled IdP accounts. -- We rely on IdP-backed WireGuard with strict, default-deny, role-based access controls to access internal applications. -- We have centralized repositories of policy and audit controls, including team onboard and offboarding and access requests; we regularly audit access to internal systems. - -## Process Controls: Network/Infrastructure Security (“InfraSec”) - -InfraSec is the practice of ensuring a hardened, minimal attack surface for components we deploy on our network. Conventionally, modern InfraSec centers on “cloud security”; of course, we are ourselves a cloud provider, which makes the job more interesting. - -- We run on our own hardware deployed in secure data centers like Equinix. -- Our platform networking runs over a WireGuard mesh with further BPF-based access controls. Everything is encrypted in transit, at multiple layers. -- Remote management is largely automated, and fully audited; remote access is done through an IdP-backed cert-based SSH system with transcript-level audit trails. -- [Fly.io](http://Fly.io) operates a large logging and metrics cluster (it’s a feature of our platform!). -- Our internal client/server services are locked down further with Mutual TLS. -- We work with upstream traffic providers to perform automated and manual DDoS mitigation. -- Compute jobs at [Fly.io](http://Fly.io) are virtualized using Firecracker, the virtualization engine developed at AWS as the engine for Lambda and Fargate. -- Customer information on databases and volumes at [Fly.io](http://Fly.io) is encrypted with the Linux LUKS block storage encryption secrets. - -## Application Security (“AppSec”) - -AppSec is the practice of ensuring software is secure by design, secured during development, secured with testing and review, and securely deployed. - -- Our software is built in memory-safe programming languages, including Rust (for our Anycast forwarding path) and Golang (for our deployment and control plane).  -- We make decisions that minimize our attack surface. Most interactions with [Fly.io](http://Fly.io) are well-described in a GraphQL API, and occur through flyctl, our open-source command-line tool. -- We perform internal code reviews with a modern, PR-based development workflow, and engage external testing firms to assess our software security. - -## Vulnerability Remediation - -Vulnerabilities that directly affect Fly.io's systems and services will be patched or otherwise remediated within a timeframe appropriate for the severity of the vulnerability, subject to the public availability of a patch or other remediation instructions. - -**Severity: Timeframe** -* Critical: 24 hours -* High: 1 week -* Medium: 1 month -* Low: 3 months -* Informational: As necessary - -If there's a severity rating that accompanies a vulnerability disclosure, we'll generally rely on that as a starting point, but may upgrade or downgrade the severity in our best judgement. - - -## SOC2 and HIPAA - -[We have our SOC2 Type I](/blog/soc2-the-screenshots-will-continue-until-security-improves/) where we've documented a bunch of these controls. Additionally, we've detailed a number of controls for folks exploring [running HIPAA-compliant applications on our platform](/docs/about/healthcare). - -## Questions? - -[Email us!](mailto:security@fly.io) \ No newline at end of file diff --git a/about/support.html.md b/about/support.html.md index f39618fa3f..1c3b6ab20c 100644 --- a/about/support.html.md +++ b/about/support.html.md @@ -2,10 +2,11 @@ title: Support layout: docs nav: firecracker -toc: false --- -An open laptop with a Fly.io sticker on it, sitting on the ground next to a coffee mug and a cute turtle. +
+ Illustration by Annie Ruygt of Frankie the hot air balloon wearing a fireman hat, putting a ladder on a tree to save a cat +
Need support? We can help! Here's where to look. @@ -25,15 +26,45 @@ All customers have access to our active [community forum](https://community.fly. Customers also frequently help each other out and share insights on running their apps on Fly.io. Especially helpful contributors can earn a special "Aeronaut" badge to signify their troubleshooting prowess, great advice, and exceptional kindness. -## Email support +## Billing and account support -**Who can use this:** Customers on [Launch, Scale, and Enterprise plans](https://fly.io/plans). +**Who can use this:** Everyone + +**Use this for:** Billing and account management issues. + +For questions about a specific invoice or account management issues, you can email us at [billing@fly.io](mailto:billing@fly.io). + +## Paid support plans + +**Find out more:** Want to learn more about our different support plans or ready to get started? Just head over to our [Support page](https://fly.io/support). We've got all the details there to help you choose what works best for you! + +### Email support + +**Who can use this:** Organizations who have purchased a Standard, Premium, or Enterprise Support package or organizations with legacy Launch or Scale plans. + +**Use this for:** Technical support for issues or questions specific to you. + +Our Standard, Premium, and Enterprise packages have access to email support. These plans come with an organization-specific address for emailing support questions, typically `@support.fly.io`. -**Use this for:** Issues or questions specific to you. +You can find your support address is in the [Fly.io dashboard](https://fly.io/dashboard). Select your organization, and look for your unique email address in the **Support** section. You'll also find a form to submit new support tickets, as well as an overview of recent support interactions. -Our Launch, Scale, and Enterprise plans have access to email support. These plans come with an organization-specific address for emailing support questions, typically <org-name>@support.fly.io. +### Support Portal -Your support address is in the [Fly.io dashboard](https://fly.io/dashboard). Select your organization, and look for your Support email address in the Billing section. +**Who can use this:** Organizations who have purchased a Standard, Premium, or Enterprise Support package or organizations with legacy Launch or Scale plans. Organizations with a Managed Postgres (MPG) database cluster also have access to our Support Portal for issues related to MPG. + +**Use this for:** Technical support for issues or questions specific to you. + +The Support Portal is a self-service portal for customers to submit support tickets, view recent support interactions, and track the status of their tickets. You can access it from your [Fly.io dashboard](https://fly.io/dashboard) by clicking the **Support** tab. + +## Managed Postgres Support + +We now offer [Managed Postgres (MPG)](https://fly.io/docs/mpg/overview/), a fully managed Postgres service with automated provisioning, daily snapshots, built-in high availability, global networking, and Prometheus-compatible metrics. MPG is currently available in a limited set of regions and supports up to 1 TB of storage, with an initial allocation cap of 500 GB. Managed Postgres customers have Support portal access for issues related to MPG. + +Note: If you need to run in unsupported regions, require more storage, or need custom configuration, you may still need to use an [unmanaged Postgres](https://fly.io/docs/postgres/getting-started/what-you-should-know/) app for now. + +## Support metrics + +The Fly.io Support team publishes our email support metrics to better set expectations regarding response times and help to you understand how we're doing. You can view these metrics in the public [Support Metrics Dashboard](https://fly.io/support). ## Working with support @@ -48,6 +79,40 @@ Here are some things to include in your ticket: - Prepending `LOG_LEVEL=debug` to any flyctl command will provide insight into the lower-level API calls being made. - If you are sending us error messages or log output, it's best to send that as plain text in the body of the email (or upload a `.txt` or `.log` file for longer outputs), rather than attached screenshots of your terminal window. -## Other questions? - -For questions about a specific invoice or account management issues, email us at [billing@fly.io](mailto:billing@fly.io). +## Scope of Support + +
+

+ This section highlights the products we actively work on and maintain, and therefore, the ones we are able to provide support for. While we can't offer direct support for your own code or the specific language or framework you use, here are some [guides to the most common frameworks](https://fly.io/docs/getting-started/get-started-by-framework/) used on Fly.io. +

+ + +
+
+
+

Supported Products

+
    +
  • **Networking**
    • +
  • **Machines** (including GPUs)
    • +
  • **Managed Postgres** (MPG)
    • +
  • **Apps**
    • +
  • **Launch/Deploy** (UI & CLI)
    • +
  • **Volumes**
    • +
  • **FKS**
    • +
  • **Security**
    • +
  • **Accounts & Billing**
    • +
  • **Extensions** (Tigris, Upstash, Depot)
    • +
  • **Monitoring** (metrics and logs)
    • +
+
+ +
+

Not Supported

+
    +
  • **LiteFS**
    • +
  • **Unmanaged Postgres**
    • +
+
+
+
+
diff --git a/app-guides/continuous-deployment-with-github-actions.html.md b/app-guides/continuous-deployment-with-github-actions.html.md deleted file mode 100644 index eee8d33077..0000000000 --- a/app-guides/continuous-deployment-with-github-actions.html.md +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Continuous Deployment with Fly.io and GitHub Actions" -layout: docs -sitemap: false -nav: firecracker -categories: - - ci - - github - - guide -priority: 10 -date: 2020-06-20 ---- - -A man writing code on a vintage desktop computer - -This guide works through setting up your app for continuous deployment to Fly.io from the app's GitHub repository. - -You'll start with an example application called [go-example](https://github.com/fly-apps/go-example). It's a simple, lightweight app that displays the Fly.io region that served the request. - -The first section is a speed-run through the steps to make the go-example app automatically deploy to Fly.io from GitHub. The next section loops back for a [longer look at each step](#a-longer-look-at-the-deployment-process). - -## Speed-run your way to continuous deployment - -1. Fork the [go-example](https://github.com/fly-apps/go-example) repository to your GitHub account. -2. Clone the new repository to your local machine. -3. Run `fly launch --no-deploy` from within the project source directory to create a new app and a `fly.toml` configuration file. -4. Type `y` to when prompted to tweak settings and enter a name for the app. Adjust other settings, such as region, as needed. Then click **Confirm Settings**. -5. Still in the project source directory, get a Fly API deploy token by running `fly tokens create deploy -x 999999h`. Copy the output. -6. Go to your newly-created repository on GitHub and select **Settings**. -7. Under **Secrets and variables**, select **Actions**, and then create a new repository secret called `FLY_API_TOKEN` with the value of the token from step 5. -8. Back in your project source directory, create `.github/workflows/fly.yml` with these contents: - - ```yaml - name: Fly Deploy - on: - push: - branches: - - master - jobs: - deploy: - name: Deploy app - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v3 - - uses: superfly/flyctl-actions/setup-flyctl@master - - run: flyctl deploy --remote-only - env: - FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} - ``` - -
- **Note:** The `go-example`’s default branch is currently `master`. If you’re using a different app, yours might be `main`. Change the `fly.yml` file accordingly. -
- -9. Commit your changes and push them up to GitHub. You should be pushing two new files: `fly.toml`, the [Fly Launch](/docs/apps/) configuration file, and `fly.yml`, the GitHub action file. - -Then the magic happens - The push triggers a deploy, and from now on whenever you push a change, the app will automatically be redeployed. - -If you want to watch the process take place, head to the repository and select the **Actions** tab, where you can view live logs of the commands running. - -## A longer look at the deployment process - -### `fly.toml` and the repository - -**Step 1** is a simple GitHub Fork; there's not a lot to say about that except that you need to do it, because you want control of the repository that you're deploying from. - -**Step 2** is just cloning the repository to your local system so that you can edit and push changes to it. - -**Steps 3 and 4** create a new app on Fly.io and a `fly.toml` configuration file to go into the repository. - -
-A note about `fly.toml` in repositories: Usually, when Fly.io ships examples, we avoid putting the `fly.toml` file in the repository by including `fly.toml` in the `.gitignore` file. And users should be creating their own `fly.toml` with the `fly launch` command. When using GitHub Actions though, you want your `fly.toml` in the repository so that the action can use it in the deployment process. -
- -### API tokens - -**Step 5** is about getting an API token. You can generate a deploy token to use to authorize a specific application. That's what `flyctl tokens create deploy -x 999999h` gives you. For a more powerful token that can manage multiple applications, run `flyctl auth token`. - -**Steps 6 and 7** make your new token available to GitHub Actions that run against your repository. You'll add the token as a secret in the repository's settings. Under the **Settings** tab, go to **Secrets and variables** and select **Actions**. Click on the green "New repository secret" button, enter the name as `FLY_API_TOKEN`, and copy the token as the secret. - -If you'd prefer an environment secret instead, then you need to list the environment you selected in your deploy step. For example: - -```yaml -deploy: - name: Deploy app - runs-on: ubuntu-latest - environment: production -``` - -### Building the workflow and deployment - -**Step 8** is the heart of the process, where you put in place a workflow. Now, GitHub has a UI which allows you to select and edit workflows, but you can also modify them as part of the repository. So you create `.github/workflows/fly.yml` - you'll likely want to `mkdir -p .github/workflows` to quickly create the directories - and load up the file with a GitHub Action recipe. - -Github Action recipe, line by line: - -```yaml -name: Fly Deploy -``` - -This sets the displayed name for the action. - -```yaml -on: - push: - branches: - - master -``` - -When should this action be run. There's lots of options but in this case, in response to any `push` to the repository's `master` branch. If your repository uses a default branch other than `master`, such as `main`, then you should change that here. - -```yaml -jobs: - deploy: - name: Deploy app - runs-on: ubuntu-latest -``` - -So an action is made up of named jobs, in this case one to deploy the application. The jobs run on a virtual machine. This section gives the "deploy" job the name "Deploy app" and tells GitHub Actions to run it on a virtual machine with the latest version of Ubuntu on it. The next part is to set up the steps needed to complete this job. - -```yaml - steps: - - uses: actions/checkout@v3 -``` - -The first step is one of the built in Actions steps. The step `uses` the `checkout@v3` action which checks out the repository into a directory on the virtual machine. You're now ready to deploy. - -```yaml - - uses: superfly/flyctl-actions/setup-flyctl@master - - run: flyctl deploy --remote-only -``` - -This step `uses` the superfly/flyctl-actions action. This is a GitHub action created by Fly.io which wraps around the `flyctl` command. The wrapper is invoked with the `deploy` argument which will take over the process of building and moving the application to the Fly.io infrastructure. It uses the settings from the `fly.toml` file to guide it and uses the `FLY_API_TOKEN` to authorize its access to the Fly.io GraphQL API. - -```yaml - env: - FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} -``` - -This pulls the API token from GitHub's secrets engine and puts it into the environmental variables passed to the action. - -**Step 9** pushes your two new files to the repository: `fly.toml`, the Fly Launch configuration file, and `fly.yml`, the GitHub action file. The push triggers your first automatic deploy. The GitHub action now triggers a redeploy each time you push changes to your repo. - -## Conclusion and further reading - -And that's the deployment process. You can, of course, leverage the GitHub Actions environment to manage more intricate deployments, interact with other applications—say to send Slack messages on completion—or whatever else you can dream up. - -**Read:** - -* [Deploy Tokens](/docs/reference/deploy-tokens/) -* [GitHub Actions Documentation](https://docs.github.com/en/actions) - -**See:** - -* [GitHub Actions for flyctl](https://github.com/superfly/flyctl-actions) - - - - - diff --git a/app-guides/custom-domains-with-fly.html.md b/app-guides/custom-domains-with-fly.html.md deleted file mode 100644 index 31a7237d58..0000000000 --- a/app-guides/custom-domains-with-fly.html.md +++ /dev/null @@ -1,432 +0,0 @@ ---- -title: Custom Domains and SSL Certificates -layout: docs -sitemap: false -nav: firecracker -categories: - - ssl - - custom domains - - guide -date: 2020-07-20 ---- - -An application's brand is often encapsulated in its domain name and that in turn is wrapped with value. So being able to configure secure custom domains is essential. - -Fly.io offers a simple command-line process for manual configuration of custom domains and a GraphQL API for people integrating Fly custom domains into their automated workflows. Here, we'll be looking at both - and answering the question, which one should you use? - -## Teaching your app about custom domains - -Your application code needs to know how to accept custom domains and adjust the responses accordingly. If you're hosting content on behalf of your users, this typically means mapping the incoming hostname to a particular ID in your application database. - -When users make requests, their browser sends a `Host` header you can use to alter the behavior of your application. When you run your app server on Fly.io directly, just get the contents of the `Host` header to identify a request. - -If you're running your application on another provider, you will need to create a proxy application (like [NGINX](/docs/app-guides/global-nginx-proxy/)) to route traffic through Fly. Your application can then use the `X-Forwarded-Host` header to determine how to handle requests. - -## Creating a custom domain on Fly.io manually - -There's a question to ask and answer. Do you want to start accepting traffic immediately on your custom domain or do you want to have your domain ready with certificates when you set it to start accepting traffic, for example when you want to cut over from another platform to Fly. - -### Accepting traffic immediately for the custom domain - -In this scenario, we want the custom domain to point to the `nginxproxy` server which will allow unencrypted IPv4 and IPv6 connections. Again, there are two ways to do this. Using DNS's CNAME capability or setting the A and AAAA records. - -#### Option I: CNAME records - -CNAME records in DNS act like a pointer. If we add a CNAME record to our custom domain that points to our proxy name `nginxproxy.fly.dev` then requests for the custom domain's IP address would return the proxy's address and clients would then lookup the IP addresses for the proxy. - -It's the quickest way to get set up, but there are catches. First, it is ever so slightly slower with that second look up. Second, it limits what you can do with the domain, especially if it's an "Apex domain" - CNAMEs are, according to DNS standards, meant to be the only record in a host's DNS records and so you can't add MX and other essential records to the DNS entry. If you aren't setting up an Apex domain, the CNAME is the quickest way to get going. - -#### Option II: A and AAAA records - -We can skip all CNAME concerns by setting A and AAAA records on your custom domain. Run `flyctl ips list` to see your app's addresses: - -```cmd -flyctl ips list -``` -```output - TYPE ADDRESS CREATED AT - v4 77.83.143.105 2020-03-02T14:59:13Z - v6 2a09:8280:1:659f:6cb7:4925:6bfd:90a3 2020-03-02T14:59:13Z -``` - -Create an A record pointing to your v4 address, and an AAAA record pointing to your v6 address. You're then free to make this an Apex domain as needed. - -#### Adding the certificate - -Once these settings are in place, you can add the custom domain to the application's certificates. If we are configuring example.com, then we would simply run: - -```cmd -flyctl certs create example.com -``` - -If you are using [dedicated IPs](https://fly.io/docs/reference/services/#dedicated-ipv4) and want to add a wildcard domain, remember to place quotes around the hostname to avoid inadvertent shell expansion: - -```cmd -flyctl certs create "*.example.com" -``` - -This will kick off the process of validating your domain and generating certificates. You can check on the progress if you run: - -```cmd -flyctl certs show example.com -``` -```output - Hostname = example.com - Configured = true - Issued = rsa,ecdsa - Certificate Authority = lets_encrypt - DNS Provider = dnsimple - DNS Validation Instructions = CNAME _acme-challenge.example.com => example.com.o055.flydns.net. - DNS Validation Hostname = _acme-challenge.example.com - DNS Validation Target = example.com.o055.flydns.net - Source = fly - Created At = 1m9s ago - Status = Ready -``` - -Configured should be true and Status will show ready when the certificates are available. The Issued field will show which types of certificates are available - RSA and/or ECDSA. Once they are issued, you'll be ready to run with your custom domain. - -### Configuring certificates before accepting traffic - -This process allows certificates to be issued before the domain is live and accepting traffic. Using a particular CNAME entry in the DNS allows Fly.io to validate the domain before issuing a certificate. - -The process starts with adding the certificate to the application like so: - -```cmd -flyctl certs create example.com -``` - -Again, if you are creating a wildcard domain, remember to place quotes around the hostname to avoid inadvertent shell expansion: - -```cmd -flyctl certs create "*.example.com" -``` - -Now we have created the certificate, we need to ask for the CNAME entry that has to be set up. For this, we run `fly certs show`: - -```cmd -flyctl certs show example.com -``` -```output - Hostname = example.com - Configured = false - Issued = - Certificate Authority = - DNS Provider = dnsimple - DNS Validation Instructions = CNAME _acme-challenge.example.com => example.com.o055.flydns.net. - DNS Validation Hostname = _acme-challenge.example.com - DNS Validation Target = example.com.o055.flydns.net - Source = fly - Created At = 0m9s ago - Status = -``` - -The specific part to focus in here are the DNS Validation fields: - -``` -DNS Validation Instructions = CNAME _acme-challenge.example.com => example.com.o055.flydns.net. - DNS Validation Hostname = _acme-challenge.example.com - DNS Validation Target = example.com.o055.flydns.net -``` - -Basically, the Validation Hostname, when looked up, should send requests to the Validation Target, a Fly-generated validation service. To do this, add the contents of the Validation Instructions to your DNS records; that is create a CNAME record which points the _acme-challenge subdomain to the Validation target. - -Once you have done that, wait as the validation and certificate issuing happens (check in with `flyctl certs check`). When complete, you'll be able to turn on the traffic whenever you are ready. You'll be able to do that either by setting the CNAME or by setting the A and AAAA records as described previously. - -### Other commands - -Finally, we should point out the other `flyctl certs` commands which you will want to use. - -* `flyctl certs list` - Lists the hostnames which have been added to an application and for which certificates may have been obtained. -* `flyctl certs check hostname` - Triggers a check on the domain validation and DNS configuration for the given host and return results in the same format as `flyctl certs show`. -* `flyctl certs delete hostname` - Removes the hostname from the application, dropping the certificates in the process. - -## Automating the certificate process - -To illustrate how to automate the certificates API, we are going to show the `flyctl` command line and the equivalent GraphQL request, wrapped in a compact easy-to-read Node application from our [fly-examples/hostnamesapi](https://github.com/fly-apps/hostnamesapi) repository. - -### GraphQL API Notes - -**Endpoints**: The Endpoint for the Fly API is `https://api.fly.io/graphql`. - -**Authentication**: All queries require an API token which be obtained by signing into the [Fly.io web dashboard](https://fly.io/dashboard/apps/), selecting **Account** ➡︎ **Settings** ➡︎ **Access Tokens** ➡︎ **Create Access Token**. Create a new token and carefully note the value; we suggest placing it in an environment variable such as `FLY_API_TOKEN` so it can be passed to applications. When used with the API, the token should be passed in an `Authorization` header with the value `Bearer `. - -**IDs and Names**: Applications can be referred to by name or by ID. Currently the Application ID and Name are interchangeable. This will change in a future semantically significant version. To see how to query for ID and Name, see [getappbyid.js](https://github.com/fly-apps/hostnamesapi/blob/master/getappbyid.js) and [getappbyname.js](https://github.com/fly-apps/hostnamesapi/blob/master/getappbyname.js) in the example repository. - -### Listing all the hosts of an application - -**With Flyctl**: `flyctl certs list` - -**With GraphQL**: The example is in the repository as [getcerts.js](https://github.com/fly-apps/hostnamesapi/blob/master/getcerts.js). This request takes the application name as a parameter. - -```graphql -query($appName: String!) { - app(name: $appName) { - certificates { - nodes { - createdAt - hostname - clientStatus - } - } - } -} -``` - -**Example output**: - -```json -{ - "app": { - "certificates": { - "nodes": [ - { - "createdAt": "2020-03-04T14:17:14Z", - "hostname": "example.com", - "clientStatus": "Ready" - }, - { - "createdAt": "2020-03-05T15:28:41Z", - "hostname": "exemplum.com", - "clientStatus": "Ready" - } - ] - } - } -} -``` - -This lists every host associated with the application. Each host may have up to two (RSA/ECDSA) certificates associated with it. See "[Reading a certificate from an application](#reading-a-certificate-from-an-application)" to see how to query the certificates associated with the hostnames. - -### Creating a certificate for an application - -**With Flyctl**: `flyctl certs create ` - -**With GraphQL**: The example is [addcert.js](https://github.com/fly-apps/hostnamesapi/blob/master/getcerts.js). This request takes the application id and hostname as parameters. - -```graphql -mutation($appId: ID!, $hostname: String!) { - addCertificate(appId: $appId, hostname: $hostname) { - certificate { - configured - acmeDnsConfigured - acmeAlpnConfigured - certificateAuthority - certificateRequestedAt - dnsProvider - dnsValidationInstructions - dnsValidationHostname - dnsValidationTarget - hostname - id - source - } - } -} -``` - - -**Example Output**: - -```json -{ - "addCertificate": { - "certificate": { - "configured": true, - "acmeDnsConfigured": true, - "acmeAlpnConfigured": true, - "certificateAuthority": "lets_encrypt", - "certificateRequestedAt": "2020-03-06T12:26:36Z", - "dnsProvider": "enom", - "dnsValidationInstructions": "CNAME _acme-challenge.codepope.wtf => example.com.o055.flydns.net.", - "dnsValidationHostname": "_acme-challenge.example.com", - "dnsValidationTarget": "example.com.o055.flydns.net", - "hostname": "example.com", - "id": "LO7FgYIy0sBZC8yuGNFKQH4QCq4ujMfJZumJCVNiQxhMq", - "source": "fly" - } - } -} -``` - -The returned data here includes all the values needed to configure DNS records for pre-traffic validation. - -### Reading a certificate from an application - -**With Flyctl**: `flyctl certs show hostname` - -**With GraphQL**: The example is [getcert.js](https://github.com/fly-apps/hostnamesapi/blob/master/getcert.js). This request takes the application name and hostname as parameters. - -```graphql -query($appName: String!, $hostname: String!) { - app(name: $appName) { - certificate(hostname: $hostname) { - configured - acmeDnsConfigured - acmeAlpnConfigured - certificateAuthority - createdAt - dnsProvider - dnsValidationInstructions - dnsValidationHostname - dnsValidationTarget - hostname - id - source - clientStatus - issued { - nodes { - type - expiresAt - } - } - } - } -} -``` - -**Example Output**: - -```json -{ - "app": { - "certificate": { - "configured": true, - "acmeDnsConfigured": true, - "acmeAlpnConfigured": true, - "certificateAuthority": "lets_encrypt", - "createdAt": "2020-03-04T17:17:39Z", - "dnsProvider": "enom", - "dnsValidationInstructions": "CNAME _acme-challenge.example.com => example.com.o055.flydns.net.", - "dnsValidationHostname": "_acme-challenge.example.com", - "dnsValidationTarget": "example.com.o055.flydns.net", - "hostname": "example.com", - "id": "4n8ikGIzjsm0s5VclwTDaSODtveu2xCZkHkKCaRilafGk", - "source": "fly", - "clientStatus": "Ready", - "issued": { - "nodes": [ - { - "type": "ecdsa", - "expiresAt": "2020-06-02T16:17:51Z" - }, - { - "type": "rsa", - "expiresAt": "2020-06-02T16:17:45Z" - } - ] - } - } - } -} -``` - -Most of the output duplicates the details from adding a certificate, including the DNS validation settings. The difference here is the `issued` section which contains an array of nodes, each of which is the type and expiry date of certificates that have been issued against this host name. Here we see that ECSDA and RSA certificates have been issued. - -### Checking a certificate - -**With Flyctl**: `flyctl certs check hostname` - -**With GraphQL**: The example is [checkcert.js](https://github.com/fly-apps/hostnamesapi/blob/master/checkcert.js). This request takes the application name and hostname as parameters. It is essentially the same as reading the certificate, but the presence of a request for the certificate's check value will start a validation process. The output is similar too. - -```graphql -query($appName: String!, $hostname: String!) { - app(name: $appName) { - certificate(hostname: $hostname) { - check - configured - acmeDnsConfigured - acmeAlpnConfigured - certificateAuthority - createdAt - dnsProvider - dnsValidationInstructions - dnsValidationHostname - dnsValidationTarget - hostname - id - source - clientStatus - issued { - nodes { - type - expiresAt - } - } - } - } -} -``` - - -### Deleting a certificate - - -**With Flyctl**: `flyctl certs delete hostname` - -**With GraphQL**: The example is [deletecert.js](https://github.com/fly-apps/hostnamesapi/blob/master/deletecert.js). This request takes the application name and hostname as parameters and will remove the hostname from the application. - -```graphql - mutation($appId: ID!, $hostname: String!) { - deleteCertificate(appId: $appId, hostname: $hostname) { - app { - name - } - certificate { - hostname - id - } - } - } -``` - -The GraphQL mutation returns the app name and the hostname and certificate id that was removed. - -**Example Output**: - -```json -{ - "deleteCertificate": { - "app": { - "name": "nginxproxy" - }, - "certificate": { - "hostname": "example.com", - "id": "6AZc4AS6ysZgHPqFg7TGzIyofewuZnToxu6lUbBi8DHGQ" - } - } -} -``` - -## Troubleshooting -### Certificate creation or validation seems to hang, stall or fail - -Let's Encrypt™ is a free, automated, and open certificate authority that Fly.io uses to issue TLS certificates for custom domains. However, Let's Encrypt™ imposes certain rate limits to ensure fair usage. If you encounter issues when creating or validating a certificate for a custom domain on Fly.io, it's possible that you've hit these rate limits. - -The following [rate limits](https://letsencrypt.org/docs/rate-limits/) from Let's Encrypt™ apply: - -* **Certificates per Registered Domain**: 50 per week -* **Duplicate Certificate limit**: 5 per week -* **Failed Validation limit**: 5 failures per hostname, per hour - -Note that certificate renewals don’t count against your **Certificates per Registered Domain** limit. - -If you encounter issues when adding or validating a certificate for a custom domain on Fly.io, you can use the following methods to troubleshoot: - -* **Use [Let's Debug](https://letsdebug.net/)**: A diagnostic tool/website to help figure out why you might not be able to issue a certificate for Let's Encrypt™. Using a set of tests, it can identify a variety of issues, including: problems with basic DNS setup, problems with nameservers, rate limiting, networking issues, CA policy issues and common website misconfigurations. -* **Wait and Retry**: If you've hit a rate limit, you'll need to wait until the rate limit window passes before you can successfully create or validate a certificate again. We don’t have a way to reset it. - -Remember, the best way to avoid hitting rate limits is to use staging environments and domains for testing and development, and to carefully plan your certificate issuance to stay within the limits. Avoid failed validation by ensuring that your DNS records are correctly configured, with no conflicting records. - -If you're building a platform on top of Fly.io, and you expect that your users will frequently delete and then recreate the same resources within a short window, consider implementing "soft delete" logic into your platform that retains the Fly.io resources for a period of time, negating the need to recreate certs frequently. - -### I use Cloudflare, and there seems to be a problem issuing or validating my Fly.io TLS certificate - -If you're using Cloudflare, you might be using their Universal SSL feature which inserts a TXT record of `_acme_challenge.` for your domain. This can interfere with our certificate validation/challenge and you should [disable](https://developers.cloudflare.com/ssl/edge-certificates/universal-ssl/disable-universal-ssl/#disable-universal-ssl-certificate) this feature. - -You can then verify that the change has propagated and the TXT record is no longer present by running `dig txt _acme-challenge. +short`. - -## Wrapping up - -You have everything you need to either hand assign a custom domain to your Fly.io application or to create your own automated multi-domain proxy. Let your ideas take flight with Fly.io. - diff --git a/app-guides/edgedb.html.md b/app-guides/edgedb.html.md index 01e4a92976..ec0177e26b 100644 --- a/app-guides/edgedb.html.md +++ b/app-guides/edgedb.html.md @@ -8,7 +8,7 @@ categories: redirect_from: /docs/getting-started/edgedb/ --- -[EdgeDB](https://www.edgedb.com) is a [graph-relational database](https://www.edgedb.com/blog/the-graph-relational-database-defined) that runs on top of Postgres and is designed as a spiritual successor to SQL. This guide explains how to perform a single-region deployment of EdgeDB with persistent storage. It will require two apps: one for Postgres and one for the EdgeDB container. +[EdgeDB](https://www.edgedb.com+external) is a [graph-relational database](https://www.edgedb.com/blog/the-graph-relational-database-defined+external) that runs on top of Postgres and is designed as a spiritual successor to SQL. This guide explains how to perform a single-region deployment of EdgeDB with persistent storage. It will require two apps: one for Postgres and one for the EdgeDB container. ## Create the Postgres app @@ -137,7 +137,7 @@ flyctl secrets set \ --app my-other-app ``` -If you've [configured a Wireguard tunnel](https://fly.io/docs/reference/private-networking/) on your local machine, you'll be able to open a REPL to your new EdgeDB instance with the `edgedb` CLI. +If you've [configured a Wireguard tunnel](https://fly.io/docs/networking/private-networking/) on your local machine, you'll be able to open a REPL to your new EdgeDB instance with the `edgedb` CLI. ```cmd edgedb --dsn edgedb://edgedb:mysecretpassword@myedgedb.internal:8080 --tls-security insecure diff --git a/app-guides/fauna.html.md b/app-guides/fauna.html.md index 7d0961e6fd..18ec1a9270 100644 --- a/app-guides/fauna.html.md +++ b/app-guides/fauna.html.md @@ -7,15 +7,15 @@ categories: - database --- -[Fauna](https://fauna.com/) is a distributed relational database with a document data model, and delivered as an API. Databases created in Fauna automatically get three geographically distributed replicas with active-active writes and ACID compliance, making it a powerful complement to Fly.io in serving low latency reads and writes for dynamic global applications. +[Fauna](https://fauna.com/+external) is a distributed relational database with a document data model, and delivered as an API. Databases created in Fauna automatically get three geographically distributed replicas with active-active writes and ACID compliance, making it a powerful complement to Fly.io in serving low latency reads and writes for dynamic global applications. ## Starter kit -Fauna provides [starter kits](https://github.com/orgs/fauna-labs/repositories?q=fly-io-starter) to help you quickly launch an app on Fly.io, using Fauna as the database backend. Before you start, read the short section below to learn how Fauna and Fly.io makes for a great combination when building low latency, dynamic global applications. +Fauna provides [starter kits](https://github.com/orgs/fauna-labs/repositories?q=fly-io-starter+external) to help you quickly launch an app on Fly.io, using Fauna as the database backend. Before you start, read the short section below to learn how Fauna and Fly.io makes for a great combination when building low latency, dynamic global applications. ## Region Groups -A Fauna Region Group refers to the locality footprint of where the replicas are located, and allows you to control where your data resides, making it possible to comply with data locality legislation, such as the General Data Protection Regulation ([GDPR](https://gdpr-info.eu/)). +A Fauna Region Group refers to the locality footprint of where the replicas are located, and allows you to control where your data resides, making it possible to comply with data locality legislation, such as the General Data Protection Regulation ([GDPR](https://gdpr-info.eu/+external)). ![Fauna Region Group](/docs/images/fauna_region_groups.png) @@ -32,7 +32,7 @@ Currently, Fauna provides 2 choices of Regions Groups, US and EU. The table belo To take full advantage of Fauna's distributed footprint, you should deploy your app on 3 Fly.io regions as well, and as close as possible to the Region Groups' replicas. -For example, let's say you created your Fauna database in the US Region Group and you used the Fauna [Python Fly.io starter kit](https://github.com/fauna-labs/python-fly-io-starter) with the default configuration to create your Fly app. Your app will be in the `sjc` region, as determined by `primary_region` in the `fly.toml` config file. You can add Machines in the other closest US regions using this command: +For example, let's say you created your Fauna database in the US Region Group and you used the Fauna [Python Fly.io starter kit](https://github.com/fauna-labs/python-fly-io-starter+external) with the default configuration to create your Fly app. Your app will be in the `sjc` region, as determined by `primary_region` in the `fly.toml` config file. You can add Machines in the other closest US regions using this command: ``` fly scale count 2 --region ord,iad diff --git a/app-guides/global-nginx-proxy.html.markerb b/app-guides/global-nginx-proxy.html.markerb index b6030905fa..07aeeb88f7 100644 --- a/app-guides/global-nginx-proxy.html.markerb +++ b/app-guides/global-nginx-proxy.html.markerb @@ -56,7 +56,7 @@ Both server sections are set to listen on port 8080. The `server_name` directive ### Deploying NGINX to Fly -We now need to create a slot for our new Fly app. Assuming you are signed up and logged in (if not see our [hands-on for getting started](/docs/hands-on/start/)), then we'll run this: +We now need to create a slot for our new Fly app. Assuming you are signed up and logged in (if not see our [hands-on for getting started](/docs/hands-on/)), then we'll run this: ```cmd flyctl apps create @@ -72,4 +72,4 @@ Our app will now be built and deployed onto the Fly infrastructure. When that's ### Choose your own adventure -With the proxy app in place, you now have a foundation for adding features "at the edge". Maybe try HTTP caching in NGINX, or look at our [OpenResty guide and learn to write Lua](/docs/app-guides/openresty-nginx-plus-lua). Or if you're feeling extra spicey, let your customers [add their own domains](/docs/app-guides/custom-domains-with-fly/) to your application. +With the proxy app in place, you now have a foundation for adding features "at the edge". Maybe try HTTP caching in NGINX, or look at our [OpenResty guide and learn to write Lua](/docs/app-guides/openresty-nginx-plus-lua). Or if you're feeling extra spicey, let your customers [add their own domains](/docs/networking/custom-domain/) to your application. diff --git a/app-guides/graphql-on-fly-with-hasura.html.markerb b/app-guides/graphql-on-fly-with-hasura.html.markerb index 62b60207d6..33335b7b92 100644 --- a/app-guides/graphql-on-fly-with-hasura.html.markerb +++ b/app-guides/graphql-on-fly-with-hasura.html.markerb @@ -39,7 +39,7 @@ For any Fly application, you need to reserve an app name. So let's create an app flyctl apps create ``` -**If you don't have `flyctl`, you'll need to install it. Check out our [Install Flyctl](/docs/hands-on/install-flyctl/) guide. If you haven't got a Fly account yet, run `flyctl auth signup` to get your own free account.** +**If you don't have `flyctl`, you'll need to install it. Check out our [Install Flyctl](/docs/flyctl/install/) guide. If you haven't got a Fly account yet, run `flyctl auth signup` to get your own free account.** You will be asked for an app name at this point. Hit return to have one auto-generated for you. You'll see the app name when the create has been completed like this: diff --git a/app-guides/livebook.html.markerb b/app-guides/livebook.html.markerb index 0df39931a9..23c1d747b1 100644 --- a/app-guides/livebook.html.markerb +++ b/app-guides/livebook.html.markerb @@ -7,7 +7,7 @@ categories: - elixir --- -In this guide we'll deploy [Livebook](https://livebook.dev/), the interactive Elixir notebook app, to a single Fly Machine using the [Fly Launch](/docs/apps/) approach. +In this guide we'll deploy [Livebook](https://livebook.dev+external), the interactive Elixir notebook app, to a single Fly Machine using the [Fly Launch](/docs/launch/) approach. We'll give Livebook a 1GB [persistent volume](/docs/volumes/) on which to store notebooks and other data. @@ -28,11 +28,11 @@ Prepare a directory for your Livebook app config and place a `fly.toml` file ins ```toml [build] - image = "ghcr.io/livebook-dev/livebook:0.11.3" + image = "ghcr.io/livebook-dev/livebook:0.13.0" # use a prebuilt Livebook Docker image [env] - ELIXIR_ERL_OPTIONS = "-proto_dist inet6_tcp" # enable ipv6 for Fly.io private networking + ERL_AFLAGS = "-proto_dist inet6_tcp" # enable ipv6 for Fly.io private networking LIVEBOOK_DATA_PATH = "/data" # /data is the mount point of the persistent volume LIVEBOOK_HOME = "/data" LIVEBOOK_ROOT_PATH = "/data" @@ -106,7 +106,7 @@ To use your Livebook, visit `https://.fly.dev` in the browser, or You can test that your notebook was saved onto the persistent volume by running `fly apps restart` and clicking "Open" on the Livebook home page to find and open it from storage. -## A note on `auto_stop_machines` and `auto_start_machines` settings +## A note on auto start and stop settings The `auto_stop_machines` and `auto_start_machines` config options are set to `true` in the app config. The Fly proxy will shut down the Machine when there's nobody connected to it. Livebook does try to save even "unsaved notebooks," but principle you can lose data if it shuts down and you have not saved your work. diff --git a/app-guides/minio.html.markerb b/app-guides/minio.html.markerb index 86b85dcdbc..de378a98cd 100644 --- a/app-guides/minio.html.markerb +++ b/app-guides/minio.html.markerb @@ -10,17 +10,17 @@ categories: date: 2022-12-11 --- -Object storage is useful when your apps need access to unstructured data like images, videos, or documents. Amazon's S3 is an obvious solution, but you can also host your own S3-compatible object storage using the AGPLv3-licensed [MinIO](https://min.io/). +Object storage is useful when your apps need access to unstructured data like images, videos, or documents. Amazon's S3 is an obvious solution, but you can also host your own S3-compatible object storage using the AGPLv3-licensed [MinIO](https://min.io/+external). We're going to set up a single instance of MinIO on Fly.io, backed by [Fly Volume storage](/docs/volumes/). -A note: it's been observed that some features of the MinIO browser console are memory-intensive enough to crash a VM with 256MB of RAM. For even small-scale production use you'll probably need a larger (non-free) VM. 512MB seems to work for poking around and uploading small objects. (For perspective, [MinIO docs recommend a minimum of 128**GB** RAM](https://min.io/docs/minio/container/operations/checklists/hardware.html#minio-hardware-checklist) for deployment at scale.) +A note: it's been observed that some features of the MinIO browser console are memory-intensive enough to crash a VM with 256MB of RAM. For even small-scale production use you'll probably need a larger (non-free) VM. 512MB seems to work for poking around and uploading small objects. (For perspective, [MinIO docs recommend a minimum of 128GB RAM](https://min.io/docs/minio/container/operations/checklists/hardware.html#minio-hardware-checklist+external) for deployment at scale.) -This is a guide to a demonstration deployment of the official standalone MinIO Docker image, and isn't tuned in any way. Check out the [MinIO](https://min.io/docs/minio/linux/index.html) [docs](https://github.com/minio/minio/tree/master/docs) for more sophisticated use. +This is a guide to a demonstration deployment of the official standalone MinIO Docker image, and isn't tuned in any way. Check out the [MinIO docs](https://min.io/docs/minio/linux/index.html+external) docs for more sophisticated use. ## Dockerfile -We'll use the official [minio/minio](https://hub.docker.com/r/minio/minio) Docker image, but with a custom start command. Here's the Dockerfile to build that: +We'll use the official [minio/minio](https://hub.docker.com/r/minio/minio+external) Docker image, but with a custom start command. Here's the Dockerfile to build that: ```docker FROM minio/minio @@ -61,7 +61,7 @@ Your app is ready! Deploy with `flyctl deploy` ## (Un)configure networking -There's no need for this app to be accessible from the public internet, if you run your object storage within the same [Fly IPV6 private network](https://fly.io/docs/reference/private-networking/) as the app(s) that want to connect to it. +There's no need for this app to be accessible from the public internet, if you run your object storage within the same [Fly IPV6 private network](https://fly.io/docs/networking/private-networking/) as the app(s) that want to connect to it. Delete the `[http_service]` block in `fly.toml`. Now the Fly proxy won't pass any requests to the app from outside your private network. @@ -74,7 +74,7 @@ fly vol create miniodata --region lhr ``` ```out -Warning! Every volume is pinned to a specific physical host. You should create two or more volumes per application to avoid downtime. Learn more at https://fly.io/docs/reference/volumes/ +Warning! Every volume is pinned to a specific physical host. You should create two or more volumes per application to avoid downtime. Learn more at https://fly.io/docs/volumes/ ? Do you still want to use the volumes feature? Yes ID: vol_6vjd0dkm5ny038mv Name: miniodata @@ -117,7 +117,7 @@ fly deploy MinIO has a web interface. It's served on the port specified by `--console-address` in the Dockerfile, which we set to `9001`. We can access this panel over our private WireGuard network. -One way to do this is to set up a [regular WireGuard tunnel](https://fly.io/docs/reference/private-networking/), and visit `http://my-minio.internal:9001` in the browser. +One way to do this is to set up a [regular WireGuard tunnel](https://fly.io/docs/networking/private-networking/), and visit `http://my-minio.internal:9001` in the browser. A simpler way is to use flyctl's user-mode WireGuard to proxy a local port to the app: @@ -134,7 +134,7 @@ Log into the admin panel with the `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD` va ## Using the `mc` MinIO Client -You can connect to your MinIO with the `mc` command-line [MinIO Client](https://min.io/docs/minio/linux/reference/minio-mc.html). +You can connect to your MinIO with the `mc` command-line [MinIO Client](https://min.io/docs/minio/linux/reference/minio-mc.html+external). MinIO listens for non-browser connections on port 9000, by default. If you're connecting using `fly proxy`, you'll have to proxy port 9000 to use `mc`. @@ -190,7 +190,7 @@ ENV MINIO_SERVER_URL="http://minio1.local:9000/" CMD [ "server", "http://minio{1...3}.local:9000/data", "--console-address", ":9001"] ``` -We're using [expansion notation](https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#sequential-hostnames) to tell MinIO that there will be 3 nodes in the cluster. +We're using [expansion notation](https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#sequential-hostnames+external) to tell MinIO that there will be 3 nodes in the cluster. ### Disk Storage @@ -245,7 +245,7 @@ fly scale count 3 ### Private networking -One of the great things about Fly.io is the [private networking](https://fly.io/docs/reference/private-networking/) we get for free. We'll use our Machines' private IPs to define hostnames in `/etc/hosts` which our nodes will use to communicate with each other. +One of the great things about Fly.io is the [private networking](https://fly.io/docs/networking/private-networking/) we get for free. We'll use our Machines' private IPs to define hostnames in `/etc/hosts` which our nodes will use to communicate with each other. Get the private IPs of your Machines: @@ -317,4 +317,4 @@ Create a bucket in one tab and then refresh the other tab. If everything is work ## What's next? -This was a basic guide for getting an instance of MinIO deployed on Fly.io, scratching the surface of what you might want to do with an S3-compatible object store. You can use your MinIO bucket storage right from the web interface. Your apps can talk to it from within the same private network. [MinIO docs](https://min.io/docs/minio/linux/index.html) cover more advanced usage. +This was a basic guide for getting an instance of MinIO deployed on Fly.io, scratching the surface of what you might want to do with an S3-compatible object store. You can use your MinIO bucket storage right from the web interface. Your apps can talk to it from within the same private network. [MinIO docs](https://min.io/docs/minio/linux/index.html+external) cover more advanced usage. diff --git a/app-guides/multiple-processes.html.md b/app-guides/multiple-processes.html.md index 37590b172c..9636248077 100644 --- a/app-guides/multiple-processes.html.md +++ b/app-guides/multiple-processes.html.md @@ -1,5 +1,5 @@ --- -title: "Running Multiple Processes Inside A Fly.io App" +title: Multiple processes inside a Fly.io app layout: docs sitemap: true toc: true @@ -10,15 +10,15 @@ categories: date: 2020-07-20 --- -This comes up a lot: how can you run multiple programs in an app on Fly.io? Recall that Fly.io apps are shipped to us in containers, usually built by Docker, and Docker has… opinions… about running multiple things in a container. +
+This guide discusses different ways to run multiple processes in your app. To learn about process groups in Fly Apps and the `[processes]` configuration in `fly.toml`, see [Run multiple process groups in an app](/docs/apps/processes/). For process group configuration with the Machines API, see the `config.processes` object in the [Machine config](docs/machines/api/machines-resource/#machine-config-object-properties). +
-Well, [we don't use Docker to run containers](https://fly.io/blog/docker-without-docker/). Your app is running in a VM, with its own kernel. You can do pretty much anything you want inside of it, including running as many programs as you like. Most of the time, the trick is just telling Docker how to do that. +This comes up a lot: how can you run multiple programs in an app on Fly.io? Recall that Fly.io apps are shipped to us in OCI images, usually built by Docker, and Docker has… opinions… about running multiple things in a container. -There are a couple different ways to run multiple processes in a Fly.io app. All of them address the first rule of programs running in Fly VM: when your entrypoint program exits, our `init` kills the VM and we start a new one. So at the end of the day, *something* has to keep running "in the foreground". +Well, [we don't use Docker to run images](https://fly.io/blog/docker-without-docker/). Your app is running in a [Machine](/docs/machines/), a fast-launching VM with its own kernel. You can do pretty much anything you want inside of it, including running as many programs as you like. Most of the time, the trick is just telling Docker how to do that. -For more information about process groups, refer to [Run multiple process groups in an app](/docs/apps/processes/). - -
Fly.io [Machines](/docs/machines) can run multiple processes natively, no need for extra configuration. [Examples here](https://community.fly.io/t/multi-process-machines/8375).
+There are a couple different ways to run multiple processes in a Fly.io app. All of them address the first rule of programs running in a Fly Machine: when your entrypoint program exits, our `init` kills the Machine and we start a new one. So at the end of the day, *something* has to keep running "in the foreground". ### Setting the scene @@ -50,9 +50,9 @@ Now, some options to actually run this stuff: ### Process groups -This is the recommended way to run multiple processes. **This method runs each process in its own VM**. Examples of running multiple processes within a single VM are found below! +[Process groups](/docs/apps/processes/) are the recommended way to run multiple processes. This method runs each process in its own Machine or group of Machines. -Fly.io has the notion of [process groups](https://community.fly.io/t/preview-multi-process-apps-get-your-workers-here/2316). You can define multiple processes in your `fly.toml`, giving each a name. Each defined process runs in its own VM within the one app. +You can define multiple process groups in your `fly.toml`, giving each a name. Each defined process runs in its own Machine or group of Machines within the one app. You can see that in action in the below (truncated) `fly.toml` file: @@ -69,7 +69,7 @@ bar_web = "/app/server -bar" script_checks = [] ``` -Here we define two processes: `web` and `bar_web`. Each command (e.g. `/app/server` and `/app/server -bar`) is setting the *command* passed to your Dockerfile *entrypoint*. That means your entrypoint needs to be able to handle a command being passed to it! +Here we define two processes: `web` and `bar_web`. The command defined for each process group is setting the *command* passed to your Dockerfile *entrypoint*. That means your entrypoint needs to be able to handle a command being passed to it! Here's an example of such an entrypoint: @@ -87,7 +87,7 @@ fi Note that under the `[[services]]` section of the `fly.toml` file, we define a service for process `web`. The `processes = [...]` array acts as a filter - it will apply only to the processes listed. -See the [announcement post](https://community.fly.io/t/preview-multi-process-apps-get-your-workers-here/2316) for more details on scaling with multiple processes. Also note that it's a bit finnicky - it's best to create *new* apps with multiple processes. Adding them on top of existing apps (or removing them from apps that are using them) may cause some confusion. We're working on it! +You can also scale Machines horizontally and vertically by process group. See the [process groups](/docs/apps/processes/) docs for details. ### Just use Bash @@ -133,7 +133,7 @@ This works well enough to connect the app to Fly.io's Anycast network, so here's The other Docker documentation suggestion: use `supervisor`. `supervisor` is an actual process manager; it'll run in the foreground and manage all our processes and, most importantly, when our processes exit, `supervisor` will (configurably) restart it. -`supervisor` has [a lot of configuration options](http://supervisord.org/configuration.html). But because we're running on Fly.io, we mostly don't care about them; the platform is doing a lot of this work for us. So the `supervisor` configuration we want is pretty simple: +`supervisor` has [a lot of configuration options](http://supervisord.org/configuration.html+external). But because we're running on Fly.io, we mostly don't care about them; the platform is doing a lot of this work for us. So the `supervisor` configuration we want is pretty simple: ```toml [supervisord] @@ -183,7 +183,7 @@ foo: /app/server bar: /app/server -bar ``` -A Procfile manager I like a lot is [overmind](https://github.com/DarthSim/overmind). `overmind` is a Go program, so it's got a small runtime, and you could, if you were fussy, build a container that just takes the `overmind` binary and none of its build deps. We won't bother, though, since we're already bringing those deps in. So: +A Procfile manager I like a lot is [overmind](https://github.com/DarthSim/overmind+external). `overmind` is a Go program, so it's got a small runtime, and you could, if you were fussy, build a container that just takes the `overmind` binary and none of its build deps. We won't bother, though, since we're already bringing those deps in. So: ```Dockerfile FROM golang @@ -221,7 +221,7 @@ There are good reasons to run multiple programs in a single container, but somet If you're running multiple heavy-weight things in a single container, you might explore just running them as separate Fly.io apps. The advantage to doing this, apart from the apps not fighting over the same CPUs and memory, is that you can scale separate apps independently, and put them in different sets of regions. -Fly.io apps can talk to each other over a [private network](https://fly.io/docs/reference/private-networking/) that's always available. They can find each other under the `.internal` top-level domain (if your apps are `foo` and `bar`, they'll be in the DNS as `foo.internal` and `bar.internal`). Because the network connection is private and encrypted, you can generally just talk back and forth without extra authentication until you know you need it; in other words, you can keep things simple. +Fly.io apps can talk to each other over a [private network](https://fly.io/docs/networking/private-networking/) that's always available. They can find each other under the `.internal` top-level domain (if your apps are `foo` and `bar`, they'll be in the DNS as `foo.internal` and `bar.internal`). Because the network connection is private and encrypted, you can generally just talk back and forth without extra authentication until you know you need it; in other words, you can keep things simple. ### Maybe you don't need multiple processes diff --git a/app-guides/mysql-on-fly.html.markerb b/app-guides/mysql-on-fly.html.markerb index 262e920897..0769f7b553 100644 --- a/app-guides/mysql-on-fly.html.markerb +++ b/app-guides/mysql-on-fly.html.markerb @@ -27,12 +27,17 @@ mkdir my-mysql cd my-mysql # Run `fly launch` to create an app -fly launch --no-deploy +# Use the --no-deploy option since we'll make changes before first deploy +# Use the --image option to specify a MySQL Docker image +fly launch --no-deploy --image mysql:8.0.37 ``` -You can name the app whatever you'd like. The name will become a hostname our application uses to connect to the database, such as `my-mysql.internal`. +Type `y` when prompted to tweak the default app settings. Then, on the Fly Launch page: -We used the `--no-deploy` option because we have some work to do before we deploy the app. +- Name the app whatever you'd like. The name will become a hostname our application uses to connect to the database, such as `my-mysql.internal`. +- If you're using MySQL 8, it's a good idea to add some additional RAM to the VM: we recommend selecting 2GB of VM Memory. + +Confirm the settings and return to your terminal. ## Configure the App @@ -43,7 +48,7 @@ Let's create a volume straight-away. If you don't create a volume, you'll lose a fly volumes create mysqldata --size 10 # gb ``` -We also need to set some secrets required by the [MySQL container](https://hub.docker.com/_/mysql): +We also need to set some secrets required by the [MySQL container](https://hub.docker.com/_/mysql+external): ```bash # Set secrets: @@ -56,82 +61,94 @@ Save these secrets somewhere, because they're not accessible after you set them. Finally, edit the `fly.toml` file that `fly launch` generated. -For apps using MySQL 8+: - ```toml -app = "my-mysql" -kill_signal = "SIGINT" -kill_timeout = 5 - -# If copy/paste'ing, adjust this -# to the region you're deploying to +# Keep your own app name +app = 'my-mysql' +# Keep your own primary region primary_region = "bos" +[build] + image = 'mysql:8' + +[[vm]] + cpu_kind = 'shared' + cpus = 1 + memory_mb = 2048 + +# The [processes] section is different for 8+, 8.4, and 5.7. Use the one that matches your version. +# Use the following for versions 8 to 8.3: [processes] -app = """--datadir /data/mysql \ - --default-authentication-plugin mysql_native_password \ - --performance-schema=OFF \ - --innodb-buffer-pool-size 64M""" + app = """--datadir /data/mysql \ + --default-authentication-plugin mysql_native_password""" + +# Uncomment and use the following for 8.4: +# [processes] +# app = """--datadir /data/mysql \ +# --mysql-native-password=ON""" +# Uncomment and use the following for 5.7: +# [processes] +# app = "--datadir /data/mysql" + +# Add the following sections for all versions [mounts] - source="mysqldata" - destination="/data" + source = "mysqldata" + destination = "/data" [env] MYSQL_DATABASE = "some_db" MYSQL_USER = "non_root_user" - -# As of 04/25/2023: -# MySQL 8.0.33 has a bug in it -# so avoid that specific version -[build] - image = "mysql:8.0.32" ``` -There's a few important things to note: +There are a few important things to note: -1. We deleted the `[[services]]` or the `[[http_service]]` block and everything under it. We don't need it! -1. We added the `[build]` section to specify an existing Docker image. We don't need to create a `Dockerfile` of our own. +1. We deleted the `[[http_service]]` block and everything under it. We don't need it! 1. In the `[env]` section, we added two not-so-secret environment variables that MySQL will need to initialize itself. * The `MYSQL_USER` here should be any user but `root`, which already exists. 1. We added the `[processes]` section for the default `app` process, which lets us pass custom commands (overriding Docker's `CMD`). - * For MySQL 8+, you'll want to use the `mysql_native_password` password plugin. - * Also for MySQL 8+, to reduce memory usage, add the performance schema and buffer pool size flags. Note that `--performance-schema=OFF` is all one string. - * **Important**: For MySQL 5.7 and MySQL 8+, we set MySQL's data directory to a subdirectory of our mounted volume. + * For MySQL 8+, you'll want to use the `mysql_native_password` authentication plugin. As shown in the fly.toml above, there are different syntax for setting this plugin based on the MySQL version. + * If you're using **MySQL 5.7**, your `app` process can be simplified as shown in the `fly.toml` example above, as it uses `mysql_native_password` [by default](https://dev.mysql.com/doc/refman/5.7/en/native-pluggable-authentication.html#:~:text=client%20programs%20use-,mysql_native_password,-by%20default.%20The). + * **MySQL 8.0.32 to 8.3** must explicitly set this with the `--default-authentication-plugin` flag. + * Starting from **MySQL 8.4**, the `--default-authentication-plugin` option is [deprecated](https://github.com/docker-library/mysql/issues/1048#issuecomment-2088836076). If you're using 8.4+, you can make use of the `--mysql-native-password` flag instead. + +
+**Important**: Set MySQL's data directory to a subdirectory of your mounted volume. Mounting a disk in Linux usually results in a `lost+found` directory being created. However, MySQL won't initialize into a data directory unless it's completely empty. That's why you need to use a subdirectory of the mounted location: `/data/mysql`. +
-If you're using MySQL 5.7, your `app` process can be simplified: -```toml -[processes] -app = "--datadir /data/mysql" +## Deploy the App + +We're now ready to deploy the MySQL app! Go ahead and run: + +```bash +fly deploy ``` -
⚠️ Mounting a disk in Linux usually results in a `lost+found` directory being created. However, MySQL won't initialize into a data directory unless it's completely empty. Therefore, we use a subdirectory of the mounted location: `/data/mysql`.
+After a minute of MySQL initializing itself (on its first run), you should now have an app running MySQL! -## Scale the App +Your other apps can access the MySQL service by its name. In my case, I would use `my-mysql.internal` as the hostname. Any app that needs to access the database should set the hostname and username as environment variables, and create a secret for the database password. -There's one more detail. MySQL 8+ has higher baseline resource demands than MySQL 5.7. +## Debugging Possible Errors -If you're using MySQL 8, it's best to add some additional RAM to the VM: +### `mysqld: Table 'mysql.plugin' doesn't exist` -```bash -# Give the vm 2GB of ram -fly scale memory 2048 -``` +This error is an indicator that the [mysql System Schema](https://dev.mysql.com/doc/refman/8.4/en/system-schema.html) inside the [MySQL data directory](https://dev.mysql.com/doc/refman/8.4/en/data-directory.html) is empty. This can happen due to different reasons. One of them is when a MySQL app is previously deployed with `--default-authentication-plugin`, and it gets re-deployed with the updated syntax `--mysql-native-password=ON`. -This isn't necessary with MySQL 5.7. +The quickest way to fix this is to revise the `--datadir` flag to a different directory from what was previously set: -## Deploy the App +```toml +# Please revise the MySQL data directory to another location -We're now ready to deploy the MySQL app! Go ahead and run: +# For example if you had: +[processes] +app = "--datadir /data/mysql" -```bash -fly deploy +# You can change the path to a different directory +[processes] +app = "--datadir /data/mysql_ ``` +Changing the [MySQL data directory](https://dev.mysql.com/doc/refman/8.4/en/data-directory.html) will allow MySQL to completely initialize the new directory with a complete `mysql system schema.` Once done, redeploy your changes with `fly deploy`. -After a minute of MySQL initializing itself (on its first run), you should now have an app running MySQL! - -Your other apps can access the MySQL service by its name. In my case, I would use `my-mysql.internal` as the hostname. Any app that needs to access the database should set the hostname and username as environment variables, and create a secret for the database password. ## Access the database from outside @@ -152,14 +169,14 @@ flyctl proxy 13306:3306 -a my-mysql Then connect to your MySQL server at `localhost:3306` and the username and password credentials from above: ```cmd -mysql -h localhost -P 3306 -u non_root_user -ppassword some_db +mysql --protocol=tcp -h localhost -P 3306 -u non_root_user -ppassword some_db ``` ## Backups We'll take a snapshot of the created volume every day. We retain 5 days of snapshots. -To restore a snapshot, make sure you have the latest version of the `fly` command, and then create a new volume using the `--snapshot-id` flag. +To restore a snapshot, make sure you have the latest version of the flyctl, and then create a new volume using the `--snapshot-id` flag. ```bash # Get a volume ID diff --git a/app-guides/nginx-at-the-edge.html.markerb b/app-guides/nginx-at-the-edge.html.markerb deleted file mode 100644 index 15cda110e8..0000000000 --- a/app-guides/nginx-at-the-edge.html.markerb +++ /dev/null @@ -1,67 +0,0 @@ ---- -title: Nginx at the edge -layout: docs -sitemap: false -nav: firecracker -date: 2020-01-10 ---- - -<%= partial "/docs/partials/obsolete_doc" %> - -Nginx is a lightweight and powerful proxy server. It runs exceptionally well close to end users, and devs can do a lot with it: everything from manage global redirect rules to full blown apps with OpenResty. - -## _Build an Nginx app_ - -Fly runs apps bundled as Docker images. Go ahead and create a working directory for this guide, we'll use `nginx-example`. - -```cmd -mkdir nginx-example -``` -```cmd -cd nginx-example -``` - -#### `./nginx.conf` - -Nginx needs a configuration file, here's a basic configuration for proxying requests to a Heroku application. Save this to your working directory as `nginx.conf`: - -```cmd -cat nginx.conf -``` -```output -user nginx; -worker_processes auto; - -error_log /dev/stdout info; # log errors stdout for Fly -pid /var/run/nginx.pid; - -events { - worker_connections 1024; -} - -http { - default_type application/octet-stream; - - log_format main '$remote_addr - $remote_user [$time_local] "$request" ' - '$status $body_bytes_sent "$http_referer" ' - '"$http_user_agent" "$http_x_forwarded_for"'; - - access_log /dev/stdout main; # log requests to stdout - sendfile on; - keepalive_timeout 65; - gzip on; - - server { - listen 80; - server_name _; - keepalive_timeout 5; - location / { - proxy_set_header X-Forwarded-Host $http_host; # set x-forwarded-host from the original host header - proxy_redirect on; #rewrite redirects - proxy_pass https://example.herokuapp.com/; #proxy to heroku ap - } - } -} -``` -#### `./Dockerfile` - diff --git a/app-guides/planetscale.html.markerb b/app-guides/planetscale.html.markerb index 8898c4db30..a569428b73 100644 --- a/app-guides/planetscale.html.markerb +++ b/app-guides/planetscale.html.markerb @@ -1,7 +1,6 @@ --- title: Fly.io and PlanetScale layout: docs -sitemap: false toc: false author: joshua categories: @@ -14,13 +13,13 @@ date: 2022-05-24 PlanetScale is a MySQL-compatible serverless database. It now supports read replicas so you can perform low-latency reads wherever your users are located. This guide walks you through setting up PlanetScale with Fly.io apps. -We'll work with a [sample Express app](https://github.com/fly-apps/nodejs-planetscale-read-replicas/) for the [Fly platform](https://fly.io) which uses a [PlanetScale](https://planetscale.com) database. +We'll work with a [sample Express app](https://github.com/fly-apps/nodejs-planetscale-read-replicas/+external) for the [Fly platform](https://fly.io) which uses a [PlanetScale](https://planetscale.com+external) database. ## Create a database The sample app assumes a database already exists. You'll need to create that database before running the app. Since read-only regions can currently only be created using the PlanetScale UI, this guide does not use their CLI. -[Sign in to PlanetScale](https://auth.planetscale.com/sign-in), creating an account if you don't already have one. You can use a free account to create a database however you will need to upgrade to their paid _Scaler_, _Team_, or _Enterprise_ [plan](https://planetscale.com/pricing) if you would like to add read-only regions. +[Sign in to PlanetScale](https://auth.planetscale.com/sign-in+external), creating an account if you don't already have one. You can use a free account to create a database however you will need to upgrade to their paid _Scaler_, _Team_, or _Enterprise_ [plan](https://planetscale.com/pricing+external) if you would like to add read-only regions. Click on the black 'New database' button and pick 'Create new database'. Give it a name and choose its region. This is your **primary** database (we'll refer to it in multiple places later on). Choose the closest region to where you will deploy your app for the best performance: @@ -87,9 +86,9 @@ Your database is now ready to be queried by our sample app! ## Run the app locally -To run it locally you will need [NodeJS](https://nodejs.org/en/download/) +To run it locally you will need [NodeJS](https://nodejs.org/en/download/+external). -1. Clone the [`nodejs-planetscale-read-replicas`](https://github.com/fly-apps/nodejs-planetscale-read-replicas) repo. +1. Clone the [`nodejs-planetscale-read-replicas`](https://github.com/fly-apps/nodejs-planetscale-read-replicas+external) repo. 2. Run `npm install` to install its dependencies. 3. Duplicate `.env.example`, naming it `.env`. 4. The application needs to know about all the available regions. There are different ways this could be done. We've opted to use a single comma-separated string of connection strings. In the `.env`, set `DATABASE_URL` as _all_ of the connection URLs you have, separated by a comma. The **first** one should be the one for your **primary** (and possibly only) database. Follow it with those for any read-only regions you've added. It will look something like this. Note the comma between them: @@ -105,7 +104,7 @@ You've successfully connected to a PlanetScale database! ## Deploy the app to Fly -If you haven't already done so, [install the Fly CLI](https://fly.io/docs/hands-on/install-flyctl/) and then [log in to Fly](https://fly.io/docs/getting-started/log-in-to-fly/). +If you haven't already done so, [install the Fly CLI](/docs/flyctl/install/) and then [log in to Fly](/docs/getting-started/sign-up-sign-in/). We've added a few files to make this sample app ready to run on Fly: a `Dockerfile`, `.dockerignore` and `fly.toml`. The `Dockerfile` tells Fly how to _package_ the application. The `.dockerignore` file specifies the files/folders we want included (else it would include ones like `.env` that we do not want). The `fly.toml` file _configures_ the application. The Fly CLI can make that for you. But since we know which variables, ports, services and so on our app needs, we made one. diff --git a/app-guides/run-a-global-image-service.html.markerb b/app-guides/run-a-global-image-service.html.markerb index 5743ea7c04..f6748c7e07 100644 --- a/app-guides/run-a-global-image-service.html.markerb +++ b/app-guides/run-a-global-image-service.html.markerb @@ -1,5 +1,5 @@ --- -title: Run a Global Image Service +title: Global Image Processing Service layout: docs sitemap: false nav: firecracker @@ -13,7 +13,7 @@ image: https://fly.io/ui/images/image-service.webp date: 2020-05-10 --- -
+
A hand holding scissors and cutting out paper for a scrapbook
@@ -27,7 +27,7 @@ With Fly you can eliminate all that administrative load by letting Fly deploy a ## _What’s Imaginary_ -Let's start with [h2non/Imaginary](https://github.com/h2non/imaginary), a fantastically useful image processing application written in Go. Once configured, it lets you process images. The images can be posted to it or you can ask it to get an image from a remote server and all of this is controlled through a URL request to the Imaginary server. +Let's start with [h2non/Imaginary](https://github.com/h2non/imaginary+external), a fantastically useful image processing application written in Go. Once configured, it lets you process images. The images can be posted to it or you can ask it to get an image from a remote server and all of this is controlled through a URL request to the Imaginary server. You can find all the source in the GitHub repository, but for this guide, we don't need to worry about that. We're going to use the developer-supplied Docker image to deploy it. @@ -58,7 +58,7 @@ The third `CMD` line contains the parameters we pass to the `imaginary` command ## _Testing Locally_ -You'll need to have Docker installed locally to test this locally. We won't go into the details of installing Docker - check out the [Docker site and guides](https://docs.docker.com/install/) for how to install. Assuming you've done this we can move on to testing. +You'll need to have Docker installed locally to test this locally. We won't go into the details of installing Docker - check out the [Docker site and guides](https://docs.docker.com/install/+external) for how to install. Assuming you've done this we can move on to testing. #### Build a Docker Instance @@ -100,7 +100,7 @@ For this you'll need flyctl, it's your CLI-powered remote control for your Fly a ### Installing `flyctl` -If you've already installed `flyctl`, move on to the next step. If not, hop over to [our installation guide](/docs/hands-on/install-flyctl/). +If you've already installed `flyctl`, move on to the next step. If not, hop over to [our installation guide](/docs/flyctl/install/). ### Sign In (or Up) for Fly diff --git a/app-guides/run-a-private-dns-over-https-service.html.markerb b/app-guides/run-a-private-dns-over-https-service.html.markerb index 0ccb973c07..08ff2ccfe8 100644 --- a/app-guides/run-a-private-dns-over-https-service.html.markerb +++ b/app-guides/run-a-private-dns-over-https-service.html.markerb @@ -9,6 +9,7 @@ tag: - dns date: 2020-04-10 --- +<%= partial "/docs/partials/obsolete_doc" %> ## _Why DNS over HTTPS?_ diff --git a/app-guides/supercronic.html.md b/app-guides/supercronic.html.md deleted file mode 100644 index 6e640226d0..0000000000 --- a/app-guides/supercronic.html.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -title: Crontab with Supercronic -layout: docs -sitemap: false -nav: firecracker -author: brad -categories: - - cron -date: 2022-11-19 -redirect_from: /docs/app-guides/superchronic/ ---- - -`crontab` is a little too opinionated for containers—it's a great little tool, but when run inside of containers it doesn't grab `ENV` vars how we'd like it to. Fortunately there's a Go binary called `supercronic` that's a drop-in replacement for containers. - -The good news is that it's pretty easy to get it running on Fly with a little bit of copy and pasting. Let's get to it. - -## Create a crontab file - -In the root of your project, add a `crontab` file. - -``` -touch ./crontab -``` - -If you need to run a job every 5 minutes, your `crontab` file would something like this: - -``` -*/5 * * * * echo "hello world!" -``` - -Check out [cron.help](https://cron.help) if you need a quick crontab syntax reference. - -## Install `supercronic` in the container - -The [latest releases for supercronic](https://github.com/aptible/supercronic/releases) are on [Github](https://github.com/aptible/supercronic), where they include copy pasta 🍝instructions for getting it in your Dockerfile. As of November 2022, the current version of `supercronic` is v0.2.1. You'll want to check the [releases page](https://github.com/aptible/supercronic/releases) for the latest version, but here's what it looks like now: - -``` -# Latest releases available at https://github.com/aptible/supercronic/releases -ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.1/supercronic-linux-amd64 \ - SUPERCRONIC=supercronic-linux-amd64 \ - SUPERCRONIC_SHA1SUM=d7f4c0886eb85249ad05ed592902fa6865bb9d70 - -RUN curl -fsSLO "$SUPERCRONIC_URL" \ - && echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \ - && chmod +x "$SUPERCRONIC" \ - && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \ - && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic - -# You might need to change this depending on where your crontab is located -COPY crontab crontab -``` - -After you put that in your Dockerfile, we're going to configure Fly to run only one instance of `supercronic`. Why? Because if you have a cronjob that does something like deliver emails to customers, you only want them to get that once. The last thing you want is them getting as many emails from your services that matches the number of instances you're running. - -## Setup a web & cron process - -At the bottom of the `fly.toml` file, add the following: - -``` -[processes] - # The command below is used to launch a Rails server; be sure to - # replace with the command you're using to launch your server. - web = "bin/rails fly:server" - cron = "supercronic /app/crontab" -``` - -Then we have to tell Fly that your `web` process matches up with a service by having this under the `[[services]].` - -``` -# Make sure there's only one `processes` entry under `[[services]]` -[[services]] - processes = ["web"] -``` - -This change tells Fly that your `web` processes that you defined under `[processes]` will be exposed to the public internet. Your `cron` process won't be exposed to the public Internet because it doesn't need to! - -## Deploy & scale the new processes - -Now that we've added `supercronic` to our `Dockerfile`, put the `crontab` at the root of our project folder, and reconfigured the `fly.toml` file, we're ready to deploy to production. - -``` -$ fly deploy -``` - -Then we'll need to scale the processes so that we only run one virtual machine container with the `cron` process. Be sure to change `web` to whatever number you had before. - -``` -$ fly scale count cron=1 web= -``` - -That's it! If all went well you now have Cron running in a `cron` process in a Fly virtual machine. When you `fly deploy` it will get the latest code changes and reboot the virtual machines. - -# Resources - -- [https://github.com/fly-apps/supercronic](https://github.com/fly-apps/supercronic) -- [https://cron.help](https://cron.help) diff --git a/app-guides/the-dreaded-port-filter.html.markerb b/app-guides/the-dreaded-port-filter.html.markerb deleted file mode 100644 index 8a04cf7e60..0000000000 --- a/app-guides/the-dreaded-port-filter.html.markerb +++ /dev/null @@ -1,54 +0,0 @@ ---- -title: "The Dreaded Port Filter" -layout: docs -sitemap: false -nav: firecracker -author: thomas -categories: - - firecracker -date: 2020-11-03 - ---- - -
-**This is here for historical purposes only, and is not current**. -You can use any port you'd like for Fly.io services, TCP or UDP. We'll probably just -take this page down soon. There's nothing you need to know here anymore! Disregard -The Dreaded Port Filter, for it is no more. -
- -Apps running on Fly.io have their own VMs, and full control of the network. But to accept incoming connections from the Internet, those apps need to register with Fly.io's Anycast CDN. Our Anycast CDN currently honors TCP connections on these ports: - -| Port | Application | -|------|-------------| - 25 | SMTP email - 53 | TCP DNS - 80 | HTTP - 443 | HTTPS - 853 | DNS-over-TLS - 5000 | We forget - 8080 | HTTP - 8443 | HTTPS - 100xx| Random stuff - 25565| Minecraft - -You light any of these ports up for your app by editing -[the [[services]] section](/docs/reference/configuration/#the-services-sections) of -its `fly.toml` configuration. - -You are doubtlessly wondering, "why *these* ports?". And the answer is: there is no good reason for this system, and we are actively at work attempting to destroy this monstrosity that we, in our own hubris, wrought upon ourselves. In the meantime: - -### If you want to use a port not on this list... - -Just ask us, on [community.fly.io](https://community.fly.io/). If you want to use a TCP port for your application, we should support it; you're doing us a favor by asking. - -### If you want to use UDP ports not on this list... - -Go right ahead. UDP doesn't have a port filter. - -### If you're concerned about ports your apps use to talk between themselves on... - -Don't be! The port filter only applies to Fly.io's Anycast network, for incoming connections from the Internet. VMs in your organization can talk to each other on any random ports they'd like, so go ahead and book up a TimescaleDB database and point your applications to it on port 5432. - - - diff --git a/app-guides/udp-and-tcp.html.markerb b/app-guides/udp-and-tcp.html.markerb deleted file mode 100644 index 83be6dec00..0000000000 --- a/app-guides/udp-and-tcp.html.markerb +++ /dev/null @@ -1,195 +0,0 @@ ---- -title: "Running Fly.io Apps On UDP and TCP" -layout: docs -sitemap: false -nav: firecracker -author: thomas -categories: - - networking -date: 2021-12-07 - ---- - -Fly.io tries to make it easy to run any containerized app on our platform, regardless of what protocols it talks over. That includes apps that rely on UDP in addition to TCP. - -Our UDP support can be a little tricky, and can catch you out, especially if your app uses both UDP and TCP --- as is the case with the most popular UDP protocols, most notably DNS. It should be easier to run things like DNS servers on Fly.io than it is! But it's not hard; there's just four gotchas, and this article walks you through them. - -## The Basic Idea - -Let's build a simple echo service. You can play the home version at [this GitHub repository](https://github.com/fly-apps/udp-echo-). We'll listen on port 5000, UDP and TCP, and just bat back whatever clients send us. - -Use `flyctl launch` to create a `fly.toml` file. Use it to wire up the ports. - -```toml -[[services]] - internal_port = 5000 - protocol = "udp" - - [[services.ports]] - port = "5000" - -[[services]] - internal_port = 5000 - protocol = "tcp" - - [[services.ports]] - port = "5000" -``` - -We're going to build this silly app in Go (don't worry, it's trivial, you'll be able to follow it even if you aren't a gopher). It's going to be vanilla socket code. But before we get started, there are four gotchas you need to know about. - -* The UDP side of your application needs to bind to the special `fly-global-services` address. But the TCP side of your application can't; it should bind to `0.0.0.0`. - -* The UDP side of your application needs to bind to the same port that is used externally. Fly will not rewrite the port; Fly only rewrites the ip address for UDP packets. - -* We support IPv6 for TCP, but not currently for UDP. - -* We swipe a couple dozen bytes from your MTU for UDP, which usually doesn't matter, but in rare cases might. - -## Let's See Some Code - -You light up UDP and TCP with standard socket code. Usually, the only fussy bit will be the `fly-global-services` bind for UDP. Here's what that looks like in Go: - -```golang -func main() { - // in other programming languages, this might look like: - // s = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP) - // s.bind("fly-global-services", port) - - udp, err := net.ListenPacket("udp", fmt.Sprintf("fly-global-services:%d", port)) - if err != nil { - log.Fatalf("can't listen on %d/udp: %s", port, err) - } - - // in other programming languages, this might look like: - // s = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) - // s.bind("0.0.0.0", port) - // s.listen() - - tcp, err := net.Listen("tcp", fmt.Sprintf(":%d", port)) - if err != nil { - log.Fatalf("can't listen on %d/tcp: %s", port, err) - } - - go handleTCP(tcp) - - handleUDP(udp) -} - -``` - -The handlers for this app are trivial and are textbook Go code; after the UDP bind, nothing else in your app will care about Fly.io. - -Here's TCP: - -```golang -func handleTCP(l net.Listener) { - for { - conn, err := l.Accept() - if err != nil { - if errors.Is(err, net.ErrClosed) { - return - } - - log.Printf("error accepting on %d/tcp: %s", port, err) - continue - } - - go handleConnection(conn) - } -} - -func handleConnection(c net.Conn) { - defer c.Close() - - lines := bufio.NewReader(c) - - for { - line, err := lines.ReadString('\n') - if err != nil { - return - } - - c.Write([]byte(line)) - } -} -``` - -And here's UDP: - -```golang -func handleUDP(c net.PacketConn) { - packet := make([]byte, 2000) - - for { - n, addr, err := c.ReadFrom(packet) - if err != nil { - if errors.Is(err, net.ErrClosed) { - return - } - - log.Printf("error reading on %d/udp: %s", port, err) - continue - } - - c.WriteTo(packet[:n], addr) - } -} -``` - -In a real Go app doing something like DNS, you might have a single thread reading messages off the UDP socket and passing them to a pool of workers over a buffered channel; you might not want to fork off a thread for every UDP message you receive. This is just an echo service, so we'll dispense with that detail. - -This program, [with a simple Dockerfile](https://github.com/fly-apps/udp-echo-/blob/main/Dockerfile) and the `fly.toml` configuration above, should just work on Fly.io. [Boot it up](https://fly.io/docs/flyctl/deploy/) and throw some UDP packets at it with netcat: - -```cmd -echo hello world | nc -w1 -u udp-echo-sample.fly.dev 5000 -``` -```output -hello world -``` - -## More On Those Four Gotchas - -### The `fly-global-services` Address - -For the most part, UDP "just works" on Fly.io, the way you'd expect it to. - -The thing about forwarding UDP that makes it different from HTTP is that UDP messages don't have HTTP headers. When a standard proxy receives an HTTP message and sends it to its target, it stashes the original source address in the headers, so the target knows who it's talking to. You can't generally do that with UDP messages. But you need to know who you're talking to in order to respond to a UDP message! - -We use ⚡️`eBPF magic`⚡️ to shuttle UDP packets across our forwarding fabric (I wrote that because it sounds cool) while transparently mapping source addresses. What that means for you is that you can generally just write a standard socket program that reads UDP messages and replies to them based on the source address of the packet. - -Here comes the first catch: the magic we're using to make this transparently work requires us to discriminate UDP traffic from TCP traffic. To receive UDP packets, your app needs to bind to the special `fly-global-services` address (that's it; that's the address). Just as importantly: **it needs to reply to messages using that address**. - -You usually need to **explicitly bind your UDP service to `fly-global-services`.** Sorry, but `0.0.0.0`, `*`, and `INADDR_ANY` generally won't do: Linux will use the wrong source address in replies if you use those. - -
If we didn't do this `fly-global-services` thing, our UDP proxying logic would also try to proxy your app's own DNS traffic; it can't otherwise tell the difference between UDP packets you're sending to respond to a user and UDP packets your app generates in the ordinary course of its business.
- -**But: TCP Can't Use `fly-global-services`** - -You can't bind your TCP service to `fly-global-services`; the TCP side of your app should probably bind to `*`. - -This is super annoying. You probably want to use the same configuration value for both UDP and TCP ports, but you have to be fussy about it on Fly.io. We're working on it! - -### UDP Won't Work Over IPv6 - -Another thing that won't impact your code that you'll need to know about is that we don't currently support UDP over IPv6. The kernel forwarding code we run to make UDP work isn't IPv6 aware. This is not OK, and we're working on it, but it's a limitation you'll run into today. - -Every Fly.io app gets an IPv6 address, and that address will work fine for the TCP side of your app, for whatever that's worth. - -### You Might Need To Be Mindful Of MTUs - -An invariant on almost every network is the maximum packet size, the MTU. Typically, the MTU is 1500 bytes. If you exceed 1500 bytes, your packets will fragment. [You don't want to fragment](https://datatracker.ietf.org/doc/html/draft-mathis-frag-harmful-00). - -Fly.io takes a bite out of all of your packets for two purposes: - -1. We forward all traffic across a [WireGuard](https://www.wireguard.com/) mesh, and WireGuard eats 60 bytes of every packet. - -2. The UDP proxy forwarding system we use grabs another dozen bytes to record the original addresses of the packets --- the source address we saw when your client's packet hit our edge. You don't see these bytes; they're stripped off the packet before it's delivered to your app. But they're used in transit. - -In most cases this won't matter to you at all. Modern UDP protocols are designed to assume they have far less than 1500 bytes to work with, because there are all sorts of weird links on the Internet with smaller MTUs. - -But if you're doing something super-custom, it's best to assume you've got something more like 1300 bytes to work with, not 1500. - -### UDP Must Listen On The Same Port Externally And Internally - -Fly will only rewrite the IP addresses for UDP packets. The destination port for inbound packets will not be rewritten. That means if your applications receives UDP traffic on port 5000 externally, the application needs to bind to `fly-global-services:5000` in order to receive that UDP traffic. diff --git a/reference/app-availability.html.markerb b/apps/app-availability.html.markerb similarity index 80% rename from reference/app-availability.html.markerb rename to apps/app-availability.html.markerb index 93f3082f8e..64a45518b3 100644 --- a/reference/app-availability.html.markerb +++ b/apps/app-availability.html.markerb @@ -1,16 +1,20 @@ --- title: App Availability and Resiliency layout: docs -sitemap: false -nav: firecracker +nav: apps +redirect_from: /docs/reference/app-availability/ --- -A resilient app is resistant to events like hardware failures or outages. Redundancy, supported by monitoring and scaling, is one of the ways to make your app more resilient, and we provide some built-in features to make this easier. +
+ Illustration by Annie Ruygt of a figure jumping from one mountian to another +
-First, it's important to understand what your app needs to take care of. +A resilient app is resistant to events like hardware failures or outages. Redundancy, supported by monitoring and scaling, is one of the ways to make your app more resilient, and we provide some built-in features to make this easier. ## What your app needs to do +First, it's important to understand what your app needs to take care of. + Since Fly Proxy does load balancing by default, you need to consider how your app handles: * session storage, so that users stay logged in when their session moves to a different server @@ -18,7 +22,7 @@ Since Fly Proxy does load balancing by default, you need to consider how your ap

A note about Fly Proxy: -Fly Proxy does a lot of work in the background, applying handlers, managing traffic and load balancing, and managing connections to services. Fly Proxy also automatically starts and stops Machines based on traffic to your app. The proxy works for [process groups with services defined](/docs/getting-started/app-services/#services-routed-with-fly-proxy). +Fly Proxy does a lot of work in the background, applying handlers, managing traffic and load balancing, and managing connections to services. Fly Proxy also automatically starts and stops Machines based on traffic to your app. The proxy works for [process groups with services defined](/docs/networking/app-services/#services-routed-with-fly-proxy).

For databases, you'll need to follow the specific recommendations for the database you're using, including how to set up clusters for replication and failover. @@ -27,7 +31,7 @@ If you're deploying Fly Postgres, our un-managed database app, then you need to When you use [Fly Volumes](/docs/volumes/) with your app, your app needs to implement replication and backups. Volumes are block devices that live on a single disk array and get mounted directly by your Machine. There's no magic. And there's no built-in replication between volumes. If your app provides a clustering mechanism with data replication, like most databases do, we recommend you take advantage of that and run multiple instances with attached volumes. When possible, we place your app's volumes in different hardware zones within a region to mitigate hardware failures. -This brings us to disaster recovery and being prepared. You should have backups of your data and understand how to recover that data if needed. Fly.io takes daily snapshots of volumes and stores them for 5 days. You can [restore a volume's data](/docs/reference/volumes/#restore-from-a-snapshot) from one of these daily snapshots. If your use case requires more frequent backups, then you'll need to set up tooling and processes to backup your volumes on the required schedule. +This brings us to disaster recovery and being prepared. You should have backups of your data and understand how to recover that data if needed. Fly.io takes daily snapshots of volumes and stores them for 5 days by default. You can [restore a volume's data](/docs/volumes/volume-manage/#restore-a-volume-from-a-snapshot) from one of these daily snapshots. If your use case requires more frequent backups, then you'll need to set up tooling and processes to backup your volumes on the required schedule. ## Fly.io features for app resiliency @@ -45,7 +49,7 @@ These features require or depend on your app configuration in the `fly.toml` fil ## Redundancy by default on first deploy -Technically, the title of this section should be "Redundancy by default on first deploy or after scaling down to zero Machines". When you deploy your app for the first time (or after scaling down to zero Machines), you get some default redundancy configurations. These settings help you to deploy a recommended setup without having to think about it. If the defaults aren't what you're looking for, then you can still configure your app and processes the way you want. Learn more about [scaling Machine CPU and RAM](/docs/apps/scale-machine/) and [scaling the number of Machines](/docs/apps/scale-count/). +You get redundancy by default on first deploy and when you deploy after scaling down to zero Machines. This feature helps you deploy a recommended setup without having to think about it. If the defaults in the following sections aren't what you're looking for, then you can still configure your app and processes the way you want. Learn more about [resilient apps and multiple Machines](/docs/blueprints/resilient-apps-multiple-machines/), [scaling Machine CPU and RAM](/docs/apps/scale-machine/), and [scaling the number of Machines](/docs/apps/scale-count/). Here's what Fly Launch (the `fly launch` or `fly deploy` command) does, based on your app configuration: @@ -79,9 +83,18 @@ The standby Machine doesn't consume resources or start up until it's needed. #### Remove a standby Machine -When an app or process group has one Machine and one standby Machine, the standby Machine is destroyed first if you scale down to one Machine. The standby Machine isn't subsequently recreated when you [scale up or back down](/docs/apps/scale-count/) using `fly scale count`. +To remove a standby Machine, run `fly status` and copy the ID of the Machine that displays the `†` symbol beside the process group name. For example: + +``` +PROCESS ID VERSION REGION STATE ROLE CHECKS LAST UPDATED +task† 784e469a443e38 43 yul stopped 2024-08-06T20:48:10Z +``` + +Then destroy the standby Machine with `fly machine destroy `. + +If you destroy a standby Machine, it is not recreated when you [scale up or back down](/docs/apps/scale-count/) using `fly scale count`. -If you add services to the process group in `fly.toml`, then the standby designation is removed from the standby Machine on the next deploy. +If you add services for a process group in `fly.toml` that previously had no services, then the standby designation is removed from the standby Machine on the next deploy. #### Create a standby Machine @@ -126,7 +139,7 @@ Default settings for new apps created using the `fly launch` command: automatica Default settings for some existing apps (or any apps that don't have these settings in `fly.toml`): automatically start but don't automatically stop Fly Machines. -Get all the details about [automatically stopping and starting Machines](/docs/apps/autostart-stop/). +Learn more about [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/) and [how to configure it](/docs/launch/autostop-autostart/). ## Health check-based routing diff --git a/apps/app-handover-guide.html.md b/apps/app-handover-guide.html.md new file mode 100644 index 0000000000..26dca973aa --- /dev/null +++ b/apps/app-handover-guide.html.md @@ -0,0 +1,59 @@ +--- +title: App handover guide +layout: docs +nav: apps +author: kcmartin +date: 2025-07-03 +--- + +
+**If you're building an app for someone else and they’re going to run it themselves on Fly.io, you’ve got two good options. You can either start development inside their Fly.io organization, or build it in yours and move it over later. Here’s how both paths work, and what can get messy during the move.** +
+ +--- + +### Option 1: Build in the customer’s org from day one + +The cleanest handoff is no handoff at all. Have the customer create their own Fly.io organization before you start building. + +1. **They create the organization and add a payment method.** Basic stuff. This is what Fly.io bills against. +1. **They invite your devs to their org.** They can do this in the dashboard under “Team → Invite Member.” You now have access to deploy apps directly into their account. +1. **You build inside their org.** Fly.io treats organizations as isolated namespaces, so there’s no awkward boundary crossing here. It’s just your team shipping code, in their environment. +1. **They remove you when it’s over.** Or not—you can stick around for maintenance if that’s part of the deal. +1. **What can get messy.** Make sure that the invited developer is not the only admin user in your customer's org. Double-check that your customer has at least two admins in their org. + +--- + +### Option 2: Build in your org, then move the app to theirs + +Sometimes it’s easier to get going inside your own org—especially if you’ve got infra and tooling already set up. When the app is ready, you move it. + +Here’s how that works: + +1. **Build as usual in your organization.** +1. **Have the customer invite one of your devs to their org.** This person needs to be in _both_ orgs to do the move. +1. **Transfer the app** +1. Use `fly apps move --org ` +1. Heads up: the move causes a couple minutes of downtime. It’s a good idea to schedule this when traffic is low, and inform your users of a maintenance window. +1. What moves automatically: + - Machines and volumes (data included) + - Secrets and environment variables + - Certificates and domain names + - LiteFS databases (as long as you’re using `$FLY_APP_NAME` for the Consul key) +1. What doesn’t move automatically: + - **Postgres** **(unmanaged and managed):** Moving Postgres databases across orgs is not supported. You’ll need to spin up a new Fly Postgres app in the target org and restore from a volume snapshot. Or you can use `pgloader` to migrate the data. + - **Upstash Redis:** This is tied to an org’s private network. You’ll need to provision a new DB in the new org. + - **Tigris buckets:** You’ll have to delete the old Tigris bucket, recreate it in the new org, and copy the data over (try `s3sync`). Don’t forget to reset your app’s secrets. + - **Sentry and other extensions**: Be sure to create fresh ones in the new org and reconfigure the app to use them. + +### Summary + +If you know up front that your customer will own the app long-term, starting in their org avoids a bunch of fiddly work later. But if you need to hand over an app from your org to theirs, Fly.io gives you enough tools to pull it off—just be ready to rewire a few things by hand. + + + +### Related reading + +[Fly.io billing](/docs/about/billing) + +[Move an app between orgs](/docs/apps/move-app-org) diff --git a/apps/autostart-stop.html.markerb b/apps/autostart-stop.html.markerb deleted file mode 100644 index 3cfe8c67db..0000000000 --- a/apps/autostart-stop.html.markerb +++ /dev/null @@ -1,121 +0,0 @@ ---- -title: Automatically stop and start Machines -objective: -layout: docs -nav: firecracker -order: 65 ---- - -Fly Machines are fast to start and stop, and you don't pay for their CPU and RAM when they're in the `stopped` state. For Fly Apps with a service configured, Fly Proxy can automatically start and stop existing Machines based on incoming requests, so that your app can meet demand without keeping extra Machines running. And if your app needs to have one or more Machines always running in your primary region, then you can set a minimum number of machines to keep running. - -The auto start and stop feature also plays well with apps whose Machines [exit from within](#stop-a-machine-by-terminating-its-main-process) when idle. If your app already shuts down when idle, then the proxy can restart it when there's traffic. - -You might also be interested in learning about [scaling the number of machines](/docs/apps/scale-count/). - -## Configure automatic start and stop - -The auto start and stop settings apply per service, so you set them within the [[[services]]](/docs/reference/configuration/#the-services-sections) or [[http_service]](/docs/reference/configuration/#the-http_service-section) sections of `fly.toml`: - -For example: - -```toml -... -[[services]] - internal_port = 8080 - protocol = "tcp" - auto_stop_machines = true - auto_start_machines = true - min_machines_running = 0 -... -``` - -Briefly, the settings are: - -* `auto_start_machines`: Whether Fly Proxy should automatically start Machines based on requests and capacity. -* `auto_stop_machines`: Whether Fly Proxy should automatically stop Machines when the app is idle for several minutes. -* `min_machines_running`: The minimum number of Machines to keep running in the primary region when `auto_stop_machines = true`. - -Concurrency limits for services affect [how automatic starts and stops work](#how-it-works). - -### Default values - -The default settings for apps that don't specify any auto start and stop settings in `fly.toml` are `auto_start_machines = true` and `auto_stop_machines = false`. - -When you create an app using the `fly launch` command, the default settings in `fly.toml` are: - -```toml -... -[http_service] - internal_port = 8080 - force_https = true - auto_stop_machines = true - auto_start_machines = true - min_machines_running = 0 -... -``` - -### Recommended settings - -If your app already [exits when idle](#stop-a-machine-by-terminating-its-main-process), then you can set `auto_start_machines = true` and `auto_stop_machines = false` to have Fly Proxy automatically restart the Machines stopped by your app. - -If your app doesn't exit when idle, then we recommend setting `auto_stop_machines` and `auto_start_machines` to the same value (either both `true` or both `false`) so that Fly Proxy doesn't stop Machines and never restart them. For example, if `auto_start_machines = false` and `auto_stop_machines = true`, then Fly Proxy automatically stops your Machines when there's low traffic but doesn't start them again. When all or most of your Machines stop, requests to your app could start failing. - -When `auto_stop_machines = true`, set `min_machines_running` to `1` or higher if you need to keep one or more Machines running all the time in your primary region. The `min_machines_running` value applies to the total number of Machines, not Machines per region. For example, if `min_machines_running = 1` and there's no traffic to your app, then Fly Proxy will stop Machines until eventually there is only one Machine running in your primary region. - -There's no "maximum machines running" setting, because the maximum number of Machines is just the total number of Machines you've created for your app. Learn more in the [How it works](#how-it-works) section. - -## How it works - -The Fly Proxy runs a process to automatically stop and start existing Fly Machines every few minutes. - -
-The automatic start and stop feature only works on existing Machines and never creates or destroys Machines for you. The maximum number of running Machines is the number of Machines you've created for your app using `fly scale count` or `fly machine clone`. Learn more about [scaling the number of Machines](/docs/apps/scale-count/). -
- -### Fly Proxy stops Machines - -When `auto_stop_machines = true` in your `fly.toml`, the proxy looks at Machines running in a single region and uses the concurrency [`soft_limit` setting](/docs/reference/configuration/#the-http_service-section) for each Machine to determine if there's excess capacity. If the proxy decides there's excess capacity, it stops exactly one Machine. The proxy repeats this process every few minutes, stopping only one Machine per region, if needed, each time. - -If you have the [`kill_signal` and `kill_timeout` options](/docs/reference/configuration/#runtime-options) configured in your `fly.toml` file, then Fly Proxy uses those settings when it stops a Machine. - -Fly Proxy determines excess capacity per region as follows: - -* If there's more than one Machine in the region: - * the proxy determines how many running Machines are over their `soft_limit` setting and then calculates excess capacity: `excess capacity = num of machines - (num machines over soft limit + 1)` - * if excess capacity is 1 or greater, then the proxy stops one Machine -* If there's only one Machine in the region: - * the proxy checks if the Machine has any traffic - * if the Machine has no traffic (a load of 0), then the proxy stops the Machine - -### Fly Proxy starts Machines - -When `auto_start_machines = true` in your `fly.toml`, the Fly Proxy restarts a Machine in the nearest region when required. - -Fly Proxy determines when to start a Machine as follows: - -* The proxy waits for a request to your app. -* If all the running Machines are above their `soft_limit` setting, then the proxy starts a stopped Machine in the nearest region (if there are any stopped Machines). -* The proxy routes the request to the newly started Machine. - -## When to stop and start Fly Machines automatically, or not - -If your app has highly variable request workloads, then you can set `auto_stop_machines` and `auto_start_machines` to `true` to manage your Fly Machines as demand decreases and increases. This could reduce costs, because you'll never have to run excess Machines to handle peak load; you'll only run, and get charged for, the number of Machines that you need. - -The difference between this feature and what's typical in autoscaling, is that it doesn't create new Machines up to a specified maximum. It automatically starts only existing Machines. For example, if you want to have a maximum of 10 Machines available to service requests, then you need to create 10 Machines for your app. - -If you need all your app's Machines to run continuously, then you can set `auto_stop_machines` and `auto_start_machines` to `false`. - -If you only need a certain number of your app's Machines to run continuously, then you can set `auto_stop_machines = true` and `min_machines_running` to `1` or higher. - -## Stop a Machine by terminating its main process - -Setting your app to automatically stop when there's excess capacity using `auto_stop_machines = true` is a substitute for when your app doesn't shut itself down automatically after a period of inactivity. If you want a custom shutdown process for your app, then you can code your app to exit from within when idle. - -Here are some examples: - -* [Shutting Down a Phoenix App When Idle](https://fly.io/phoenix-files/shut-down-idle-phoenix-app/): a post by Chris McCord on adding a task to an Elixir app's supervision tree that shuts down the Erlang runtime when there are no active connections. -* For Rails apps, the `dockerfile-rails` generator provides a [--max-idle](https://github.com/rubys/dockerfile-rails#addremove-a-feature) option that exits after _n_ seconds of inactivity. -* [A Tired Proxy in Go](https://github.com/superfly/tired-proxy) used in [Building an In-Browser IDE the Hard Way](https://fly.io/blog/remote-ide-machines/). [There's a community fork with more recent updates](https://community.fly.io/t/improved-tired-proxy-for-use-with-fly-machines/10584). -* A minimal demo app in TypeScript/Remix: [code](https://github.com/fly-apps/autoscale-to-zero-demo) & [demo](https://autoscale-to-zero-demo.fly.dev/). - -As of `flyctl` v0.0.520, Fly Postgres [supports this too](https://community.fly.io/t/scale-to-zero-postgres-for-hobby-projects/12212). diff --git a/reference/build-secrets.html.md b/apps/build-secrets.html.md similarity index 88% rename from reference/build-secrets.html.md rename to apps/build-secrets.html.md index 344cd82a9e..4828196696 100644 --- a/reference/build-secrets.html.md +++ b/apps/build-secrets.html.md @@ -1,11 +1,11 @@ --- title: Build Secrets layout: docs -sitemap: false -nav: firecracker +nav: apps +redirect_from: /docs/reference/build-secrets/ --- -You can set [secrets](/docs/reference/secrets/) for your applications, but these are only available at _run-time_. They aren't available when building your Docker image without a little extra work. +You can set [secrets](/docs/apps/secrets/) for your applications, but these are only available at _run-time_. They aren't available when building your Docker image without a little extra work. To make a secret available at build time, we'll use [Docker secrets](https://docs.docker.com/develop/develop-images/build_enhancements/). @@ -49,11 +49,11 @@ echo -n "secret_value" > mysecret.txt docker build --secret id=MY_SUPER_SECRET,src=mysecret.txt . ``` -## Automate the inclusion of build secrets using an ephemeral machine +## Automate the inclusion of build secrets using an ephemeral Machine The above requires you to have access to the values of secrets and to provide those values on the command line. If this poses a problem, an -alternative may be to create an ephemeral machine using the `fly console` +alternative may be to create an ephemeral Machine using the `fly console` command to do the deployment. An example Dockerfile you can use for that purpose: diff --git a/apps/concurrency.html.markerb b/apps/concurrency.html.markerb new file mode 100644 index 0000000000..bfc57861c0 --- /dev/null +++ b/apps/concurrency.html.markerb @@ -0,0 +1,50 @@ +--- +title: Guidelines for concurrency settings +layout: docs +nav: apps +redirect_from: /docs/reference/concurrency/ +--- + +Concurrency settings are used by Fly Proxy for important things like [load balancing](/docs/reference/load-balancing/#load-balancing-strategy) and [autostop/autostart](/docs/launch/autostop-autostart/) for Machines. + +The following concurrency settings apply per Machine and per service in your app: + +`type`: Use `connections` or `requests` as the metric for app load concurrency. + +`hard_limit`: At or above this number, no new traffic goes to the Machine. + +`soft_limit`: At or above this number, traffic to the Machine is deprioritized. Traffic is only routed to the Machine if all Machines reach the soft limit. + +For Fly Launch apps, configure concurrency settings in the [[http_service.concurrency]](/docs/reference/configuration/#http_service-concurrency) or [[services.concurrency]](/docs/reference/configuration/#services-concurrency) sub-sections of your `fly.toml` file. If you're using the Machines API, then concurrency settings are per `service` in [the `config` object](/docs/machines/api/machines-resource/#machine-config-object-properties). + +## Default settings + +The default concurrency settings when not specified are a `soft_limit` of 20 and no `hard_limit`. Many apps can handle a few hundred concurrent connections or more. You can try out different concurrency settings and see how it affects performance and load balancing. + +`connections` is the default concurrency type. + +## How Fly Proxy handles concurrency + +Fly Proxy doesn't consider the exact number of concurrent connections or requests. From the proxy's point of view a Machine is handling between 0 and the soft limit, handling between the soft limit and the hard limit, or is at the hard limit. Along with region, this is how the proxy decides which Machine gets traffic. + +The proxy's view of a Machine's load is affected when the gap between the soft limit and the hard limit is small and/or the app's concurrent connections or requests oscillate between them more frequently. In that case, the proxy routes according to the settings, but the thresholds change too quickly and requests or connections end up being re-routed. + +General caveats: + +- If the limits are too high, then your Machines might not be able to process that much concurrently and your app could crash. +- If the limits are too low, then the proxy is artificially limiting what your app can process. +- If the soft and hard limit are too close, then there might not be enough “time” for the proxy to decide to load balance and the result could be multiple retries. + +## Connections or requests + +The decision to use `connections` or `requests` for concurrency depends on the type of app and its load. + +`requests` are HTTP requests and the recommended concurrency type for web services. Using `requests` for concurrency can prevent too many connections to your app and reduce latency; the proxy can temporarily pool and reuse connections. + +`connections` are TCP connections. Multiple requests can be sent over a connection, so you need to consider how your app handles that. If you use `connections` for web services, then the proxy opens a new connection for each HTTP request, which is why `requests` is a better setting for HTTP apps. + +## Concurrency limit tuning tips + +When tuning concurrency, try setting a relatively high `hard_limit`, or leave it unset to have no hard limit. If you do want to set a `hard_limit` to have more control over load balancing, then you might have to do an initial benchmark to estimate the maximum number of concurrent connections or requests that your app can handle. Then tune the `soft_limit` and [create more Machines](/docs/blueprints/resilient-apps-multiple-machines/) to optimize autostop/autostart and load balancing. Once your app is getting real-world traffic, you can continue to monitor your app and adjust the `soft_limit` further to suit your workload. + +Want a more detailed guide to concurrency limits? See this blueprint: [Setting Hard and Soft Concurrency Limits on Fly.io](/docs/blueprints/setting-concurrency-limits/). diff --git a/apps/custom-domain.html.markerb b/apps/custom-domain.html.markerb deleted file mode 100644 index c74988838a..0000000000 --- a/apps/custom-domain.html.markerb +++ /dev/null @@ -1,83 +0,0 @@ ---- -title: Use a custom domain -layout: docs -nav: firecracker -order: 35 ---- - -When you create a Fly App, it is automatically given a `fly.dev` sub-domain, based on the app's name. This is great for testing, but when you want to go to full production you'll want your application to appear on your own domain and have HTTPS set up for you as it is with your `.fly.dev` domain. That's where the `fly certs` command comes in. But let's step back before we set up the TLS certificate, to the first step: directing traffic to your site. - -## Set a CNAME record - -The simplest option for directing traffic to your site is to create a CNAME record for your custom domain that points at your `.fly.dev` host. For example, if you have a custom domain called `example.com` and an app called `exemplum`, then you can create a CNAME record for `example.com`'s DNS that would look like: - -``` -CNAME @ exemplum.fly.dev -``` - -You'll need to configure this with your DNS provider. - -Now, accessing `example.com` will tell the DNS system to look up `exemplum.fly.dev` and return its results. - -## Set the A record - -The other option is slightly more complicated because it uses the IP address of the app, rather than its DNS name. The upside is that it is slightly faster. - -To start, you need the Fly IP address of your deployed application. To get that, use the `fly ips list` command. - -You'll need to configure the A record with your DNS provider. You need to add in an "A Record" for your domain that points to the IP address. Once this is done and propagated through the DNS system, you should be able to connect over unencrypted HTTP to using the domain name. Continuing the preceding example, that's the domain name: `http://example.com`. - -## Get certified - -To enable HTTPS on the domain, you need to get a certificate. Fly.io does that for you automatically. - -It starts with creating a certificate for your custom domain with the `fly certs add` command. For example: - -```cmd -fly certs add example.com -``` -```output - Hostname = example.com - Configured = true - Issued = - Certificate Authority = lets_encrypt - DNS Provider = enom - DNS Validation Instructions = - DNS Validation Hostname = - DNS Validation Target = example.com.5xzw.flydns.net - Source = fly - Created At = 0001-01-01T00:00:00Z - Status = -``` - -Running `fly certs add` starts the process of getting a certificate. - -Run `fly certs show` to get the details needed for your next step. For example: - -```cmd -fly certs show example.com -``` -```output - Hostname = example.com - Configured = true - Issued = ecdsa, rsa - Certificate Authority = lets_encrypt - DNS Provider = enom - DNS Validation Instructions = CNAME _acme-challenge.example.com => example.com.5xzw.flydns.net. - DNS Validation Hostname = _acme-challenge.example.com - DNS Validation Target = example.com.5xzw.flydns.net - Source = fly - Created At = 1m24s ago - Status = Ready -``` - -The **DNS Validation Instructions** are an optional next step. For a short time (minutes) after we start the process of generating -the first-ever certificate for your site, trying to load that site with an HTTPS URL will generate errors. If you'd like to make sure -those errors aren't ever visible, you can use a DNS challenge to pre-generate the certificate. - -To do that, you need to create a `CNAME` DNS record for a subdomain, `_acme-challenge`, of your domain (the DNS Validation host name) -and point it at the DNS Validation Target. The process will depend on your DNS provider. Once complete, and the updated DNS data has propagated, that domain will be queried and confirm you have control of it. Certificates will be generated and installed and you'll be able to access your custom domain. - -## Related topics - -For a more detailed example of configuring custom domains, see the [Custom domains and SSL certificates](/docs/app-guides/custom-domains-with-fly/) guide. diff --git a/apps/delete.html.markerb b/apps/delete.html.markerb index c33e2f9008..2bd5cc117e 100644 --- a/apps/delete.html.markerb +++ b/apps/delete.html.markerb @@ -1,15 +1,13 @@ --- -title: Delete a Fly App -objective: +title: Delete an app layout: docs -nav: firecracker -order: 110 +nav: apps --- -You can scale an App right down to zero Machines if you like. But if you're done with an App forever, you can delete it using `fly apps destroy `. +You can scale an app right down to zero Machines if you like. But if you're done with a Fly App forever, you can delete it using `fly apps destroy `. -Once you destroy an App, you can no longer access any part of it, and you'll no longer be charged for any part of it either. This includes Machines, Volumes, IP addresses, Secrets, Docker images, and so on. +Once you destroy an app, you can no longer access any part of it, and you'll no longer be charged for any part of it either. This includes Fly Machines, Fly Volumes, IP addresses, secrets, Docker images, and so on.
-Here's a potential footgun: When you delete an App, you delete its Volumes. After you delete a Volume, you can't access the Volume snapshots. So if you want the data from an app's persistent storage, then get the data before you delete the app. +Warning: When you destroy an app, you delete its volumes and volume snapshots. If you want the data from an app's persistent storage, then get the data before you destroy the app.
\ No newline at end of file diff --git a/apps/deploy.html.markerb b/apps/deploy.html.markerb deleted file mode 100644 index 3182177997..0000000000 --- a/apps/deploy.html.markerb +++ /dev/null @@ -1,101 +0,0 @@ ---- -title: Deploy a Fly App -layout: docs -nav: firecracker -order: 30 ---- - -The `fly deploy` flyctl command builds your Fly App and spins it up on one or more Fly Machines, applying the configuration specified in a local `fly.toml` file. - -Whether you've run [`fly launch`](/docs/apps/launch/) with the `--no-deploy` option, or want to apply changes to your deployed Fly App's source or configuration, you can make a new release by running - -```cmd -fly deploy -``` - -from the source directory of your project, where any app source code, project config, `fly.toml`, and Dockerfile are located. - -If you haven't deployed the app before, `fly deploy` creates one or two Machines for every [process group](/docs/apps/processes/) defined in the app's `fly.toml` file, depending on whether there are services and volumes configured. If you haven't explicitly defined any process groups, the first deployment gives you two Machines for redundancy. For more information about when Fly Launch creates one Machine or two Machines, refer to [Redundancy by default on first deploy](/docs/reference/app-availability/#redundancy-by-default-on-first-deploy). - -Subsequent deployments update the app's Machines as a group, with the latest local changes to the app's source and configuration. It's possible for a Fly App to have standalone Machines that aren't managed by `fly deploy`. - - -## Configuration - -flyctl will look for a `fly.toml` file to find the name of the app to operate on, and for an app configuration to apply during deployment. Use `fly deploy -a ` to override the app name given in `fly.toml` and deploy the project in the current working directory to a different existing Fly App. - -There are [a number of other options](/docs/flyctl/deploy/) you can use with the `fly deploy` command. - -## The build - -`fly deploy` builds, or gets, the Docker image for the app, and refreshes Machines with the latest changes. - -Here's how `fly deploy` determines how to get the app's Docker image: - -1. If an image is specified, either with the `--image` option or in the `[build]` section of `fly.toml`, use that image, regardless of the presence of a Dockerfile in the working directory or the use of the `--dockerfile` option. -2. Otherwise, check the [`[build]` section of `fly.toml`](/docs/reference/configuration/#the-build-section) and use the method specified there, whether it's a Dockerfile or a buildpack (but don't use buildpacks if you don't have to; they're brittle, bloated, and prone to change). -3. Otherwise, if the `--dockerfile` flag supplies a path to a Dockerfile, use that Dockerfile to build the image. You read that right. The `--dockerfile` flag is looked at _after_ the `[build]` section of the config file. This will hopefully change soon. -4. Otherwise, if there's a `Dockerfile` (named exactly `Dockerfile` or `dockerfile`) in the local working directory, use that Dockerfile for the build. - -## IP addresses - -If the app to be deployed is configured with an eligible HTTP `[[services]]` section, and it does not yet have [public IP addresses](/docs/reference/services/), `fly deploy` provisions a public IPv6 and a shared public IPv4 Anycast address. - -## Machines not managed by Fly Launch - -Machines created using `fly deploy` (or as part of a deployment during `fly launch`), or by `fly clone`ing such a Machine, carry a piece of metadata marking them as belonging to Fly Launch. These machines are updated as a group on all subsequent `fly deploy` commands, as are Machines that existed on a Machines App at the moment that it was migrated to a V2 App. - -New Machines created using `fly machine run` don't have the V2 Apps metadata, and are not automatically managed by Fly Launch (`fly deploy`), so these Machines can have their own configuration different from that of the App, and can even be based on a different Docker image. - -## Volume mounts and `fly deploy` - -If a Machine has a mounted [volume](/docs/volumes/), `fly deploy` can't be used to mount a different one. You can change the mount point at which the volume's data is available in the Machine's file system, though. This is configured in the [`[mounts]` section](/docs/reference/configuration/#the-mounts-section) of `fly.toml`. - -Learn more about [using Fly Volumes](/docs/volumes/). - -## `fly deploy` configuration in `fly.toml` - -Configure the following deployment behavior in the [`[deploy]` section](/docs/reference/configuration/#the-deploy-section) of `fly.toml`. - -### Release commands -You can run a one-off [release command](/docs/reference/configuration/#run-one-off-commands-before-releasing-a-deployment) in a temporary VM—using the successfully built release—before that release is deployed. This is good for, e.g., running database migrations. Keep in mind that these temporary VMs do not have access to any volumes you may have mounted in your apps. - -### Deployment strategy - -You can specify one of the following deployment strategies: - -* `rolling` (default): wait for each Machine to be successfully deployed before starting the update of the next one -* `immediate`: bring all Machines down for update at once -* `canary`: boot a single new Machine, verify its health, and then proceed with a rolling restart strategy -* `bluegreen`: boot a new Machine alongside each running Machine in the same region, and migrate traffic to the new Machines only once all the new Machines pass health checks - -Learn more about [setting a deployment strategy](/docs/reference/configuration/#picking-a-deployment-strategy). - - -```cmd -fly deploy --strategy canary -``` -```out -... -==> Building image -Searching for image 'flyio/hellofly:latest' remotely... -image found: img_z1nr0lpjz9v5q98w - -Watch your app at https://fly.io/apps/aged-water-8803/monitoring - -Creating canary machine for group app - Machine 328745da023e85 [app] update finished: success -Canary machines successfully created and healthy, destroying before continuing -machine 328745da023e85 was found and is currently in created state, attempting to destroy... -Updating existing machines in 'example-app-8803' with canary strategy - [1/2] Machine 3287457df77785 [app] update finished: success - [2/2] Machine 91857266c41638 [app] update finished: success - Finished deploying -... -``` - -## Not all changes require a new App release - -You can [add an IP address](/docs/reference/services/#ip-addresses) to an App, for example, without redeploying. - -Adding a [secret](/docs/reference/secrets/) to the App does require a restart, so `fly secrets set` triggers a new deployment. diff --git a/apps/fine-tune-apps.html.markerb b/apps/fine-tune-apps.html.markerb new file mode 100644 index 0000000000..5e037ec6f7 --- /dev/null +++ b/apps/fine-tune-apps.html.markerb @@ -0,0 +1,96 @@ +--- +title: Tips to fine-tune your app on Fly.io +layout: docs +nav: apps +redirect_from: /docs/reference/fine-tune-apps/ +--- + +It’s safe to assume Fly.io can support thousands of requests or connections per second and can keep hundreds of thousands of them live at any given moment. This is even truer when the clients connect from multiple geographic locations. + +That settled, you can still use the following guidelines to gather data to fine-tune and optimize your app and configuration to run on Fly.io. + +## Fly.io metrics + +We have various ways to access metrics for Fly Proxy, your app, and Fly Machines. Learn more about [Fly Metrics](/docs/reference/metrics/). + +- Edge HTTP response time is the total time it takes to respond with the first HTTP response headers. +- App HTTP response time is the same, but only considers your app’s response time, not the overhead of routing, load balancing, retries, and so on. + +CPU and memory resource usage are interesting, but are rarely the culprit unless you have everything else in your benchmark right. The application logic often is often the issue when it comes to excessive CPU and memory usage. + +## Routing + +Check which [Fly.io region](/docs/reference/regions/) you're reaching. + +The region you reach should be the fastest region for you, which may not be the closest geographically. You can use the debug.fly.dev server to check your region: + +``` +curl debug.fly.dev +``` + +Use `traceroute` or `mtr` to see the routing to Fly.io. + +For example: + +``` +traceroute debug.fly.dev +``` + +Check the base latency for the region you’re reaching: + +``` +ping debug.fly.dev +``` + +
+**Important**: Do you have a stable internet connection? `ping` can jitter quite a bit on bad routes. +
+ +## Application logic and performance + +Some common culprits of excessive CPU and memory usage are: + +- Using a single thread +- Blocking +- Various misconfigurations, for example, wrong or missing environment variables + +### HTTP response times + +Fly.io HTTP response time metrics measure how long it takes for your app to start responding with headers, not the response body. + +### Multithreading + +If your app has multiple threads, make sure that contention isn't an issue. You can also [scale CPU resources](/docs/apps/scale-machine/) as needed. + +## Fly.io configuration + +Some configuration settings that are more likely to affect app performance and benchmarking. + +### Machine sizing + +Right-size Machines to use more parallel processing. Learn more about [Machine sizing](/docs/machines/guides-examples/machine-sizing/#general-rules). + +### Concurrency settings for connections or requests + +Refer to our guidelines for [concurrency settings](/docs/reference/concurrency/). + +### Auto stop and start feature + +[Auto stop and start](/docs/launch/autostop-autostart/) can be turned off if you want to test the performance of your app without that variable. + +If you don’t turn off auto stop and start, then you’ll likely have some pretty high p99 values due to the initial cost of Fly Proxy queueing connections while waiting for the Machine to start. + +## Distributed systems + +Fly.io has rate limits and hard limits outside of the control of each app; these limits are per edge and per service. If you test from a single location, then you'll hit our rate limits much faster than if you test from multiple locations. + +Pay attention to how your benchmarking nodes are routing. Inefficient routing adds latency and will skew your results. If incorrect routing is causing issues with your app testing or your production apps, then post in community or contact support (depending on you plan) and we'll look into it with our network providers. + +## Challenges with benchmarking + +You probably won’t be able to realistically benchmark your app for a combination of reasons, including: + +- Users will have a large variety of ISPs and be routed differently based on ad hoc peering agreements, sometimes in non-optimal ways. +- It’s difficult to remove the Anycast routing factor from benchmarking; you have to live with it. + +In some cases, a better option might be to send part of your real traffic to Fly.io and then compare metrics. diff --git a/apps/going-to-production.html.markerb b/apps/going-to-production.html.markerb new file mode 100644 index 0000000000..7a173a7cc6 --- /dev/null +++ b/apps/going-to-production.html.markerb @@ -0,0 +1,130 @@ +--- +title: Going to production checklist +layout: docs +nav: apps +redirect_from: + - /docs/going-to-production/ + - /docs/going-to-production/the-basics/ + - /docs/going-to-production/the-basics/production-databases/ + - /docs/reference/going-to-production/ +--- + +Use this checklist to help you set up a production environment on Fly.io. + +
+ Illustration by Annie Ruygt of Frankie the hot air balloon waving to a bird sitting on a hour roof +
+ +## Overview + +Moving an app from staging to production can expose unexpected failure modes: security holes, performance and scaling issues, or data loss. This checklist is meant to catch common pitfalls for apps on Fly.io, but it’s not a guarantee of production readiness. Not every item here will apply to your app, and you may have additional requirements that aren't listed. Use this as a foundation and adapt it to your needs. Think of this list as a scaffold, not a silver bullet. + +
+**Important:** The checklist is not exhaustive and does not guarantee production-readiness for your app. Apps can have unique requirements for production depending on the framework and type of app. Some items won't be applicable and there may be other considerations not listed here; you'll need to decide which checklist items work for your app. +
+ +## Security + +<%= render ChecklistComponent.new( + items: [ + { id: "sso", title: "Set up single sign-on for organizations", description: "Enable SSO on your organization to take advantage of Google or GitHub authentication security. Learn more about [Single sign-on for organizations](/docs/security/sso/)." }, + { id: "isolation", title: "Isolate staging and production environments", description: "Use organizations to limit access to your production environment. Read this guide: [Staging and production isolation](/docs/blueprints/staging-prod-isolation/)." }, + { id: "least-privilege", title: "Enforce least privilege access", description: "Use access tokens to allow only the minimum access level required by team members to your organization, apps, and Machines. Understand [access tokens](https://fly.io/docs/security/tokens/)." }, + { id: "secrets", title: "Protect sensitive information", description: "Set secrets to store sensitive data and make them available as environment variables to your app. Read about [Secrets and Fly Apps](/docs/apps/secrets/)." }, + { id: "exposure", title: "Make sure private services are not exposed", description: "Check that your private apps with services don't have public IP addresses. Run `fly ips list` and use `fly ips release` to release unnecessary public IPs. More detail is available in this `flyctl` reference: [`fly ips` commands](/docs/flyctl/ips/). Assign private apps a [Flycast address](https://fly.io/docs/networking/flycast/) instead." }, + { id: "arcjet", title: "Use Arcjet application security for JavaScript apps", description: "Secure your app with rate limiting, bot protection, email validation, and defense against common attacks through our extension partner Arcjet. Read more about [Application Security by Arcjet](/docs/security/arcjet/)." } + ], + c: params[:c] || "", + o: params[:o] || "", + h: params[:h] || "" +) %> + +## Databases + +<%= render ChecklistComponent.new( + items: [ + { id: "production-grade-postgres", title: "Use Managed Postgres", description: "We recommend using [Fly.io's Managed Postgres](/docs/mpg/), our fully-managed database service that handles all aspects of running production PostgreSQL."}, + { id: "test-backups", title: "Practice your disaster recovery plan", description: "Practice restoring your managed Postgres database from a backup before you actually need to. You can do this anytime from the Managed Postgres dashboard." } + ], + c: params[:c] || "", + o: params[:o] || "", + h: params[:h] || "" +) %> + +## App performance + +<%= render ChecklistComponent.new( + items: [ + { id: "machine-sizing", title: "Get Machine sizing right", description: "Most conventional production web apps require [performance CPUs](/docs/machines/cpu-performance/). Also make sure you have enough RAM for your app and/or enable [swapping to disk](/docs/reference/configuration/#swap_size_mb-option) to deal with brief spikes in memory use. Find out more details in our [Machine sizing guide](/docs/machines/guides-examples/machine-sizing/)."}, + { id: "fine-tune-app", title: "Fine-tune your app", description: "Learn about optimizing your app on Fly.io. Read these tips to [fine-tune your app on Fly.io](/docs/apps/fine-tune-apps/)."} + ], + c: params[:c] || "", + o: params[:o] || "", + h: params[:h] || "" +) %> + +## Availability, resiliency, and costs + +<%= render ChecklistComponent.new( + items: [ + { id: "multiple-machines", title: "Use multiple Machines for resiliency", description: "Make your app resilient to single-host failures with multiple Machines that stay stopped until you need them. Learn more in our guide: [Resilient apps use multiple Machines](/docs/blueprints/resilient-apps-multiple-machines/)."}, + { id: "add-regions", title: "Scale your app into more regions", description: "Scale your app in multiple regions closest to your app's users. Find out how to [Scale an app's regions](/docs/launch/scale-count/#scale-an-apps-regions)."}, + { id: "autostop-autostart", title: "Use autostop/autostart to reduce costs", description: "Autostop/autostart lets you stop or suspend Machines when there's low traffic, saving on resource usage and costs. You get autostop/autostart by default with a new app, but you can configure it to optimize for your use case. Find out more: [Autostop/autostart Machines](/docs/launch/autostop-autostart/)."}, + { id: "autoscale-by-metric", title: "Set up autoscaling by metric to reduce costs", description: "For apps that aren't running web services, use the autoscaler app to scale your app's Machines based on any metric, saving on resource usage and costs. Learn how to [Autoscale based on metrics](/docs/launch/autoscale-by-metric/)."} + ], + c: params[:c] || "", + o: params[:o] || "", + h: params[:h] || "" +) %> + +## Networking + +<%= render ChecklistComponent.new( + items: [ + { id: "custom-domain", title: "Set up a custom domain", description: "Configure a certificate for your domain. Learn how to [use a custom domain](/docs/networking/custom-domain/)."}, + { id: "ipv4", title: "Consider using a dedicated IPv4 address", description: "Completely eliminate the chance of blacklisted spammers causing problems for your app. There is a small [added cost](/docs/about/pricing/#anycast-ip-addresses) for dedicated IPv4 addresses. Read more about [Dedicated IPv4](/docs/networking/services/#dedicated-ipv4)."}, + { id: "flycast", title: "Set up Flycast for private apps", description: "If you haven't already done so, give your private apps a Flycast address to communicate with them entirely on your private network. Find out about [Flycast - Private Fly Proxy services](https://fly.io/docs/networking/flycast/)."} + ], + c: params[:c] || "", + o: params[:o] || "", + h: params[:h] || "" +) %> + + +## Monitoring + +<%= render ChecklistComponent.new( + items: [ + + { id: "metrics", title: "Monitor your app with fully-managed metrics", description: "Use managed Prometheus and managed Grafana dashboards to monitor your app. Read about [Metrics on Fly.io](/docs/monitoring/metrics/)."}, + { id: "sentry", title: "Use Sentry for Error tracking", description: "Our extension partner Sentry provides an application monitoring platform that helps you identify and fix software problems before they impact your users. Fly.io organizations get a year's worth of [Sentry Team Plan](https://fly.io/docs/monitoring/sentry/#sentry-plan-details) credits. Read how to configure [Application Monitoring by Sentry](/docs/monitoring/sentry/)."}, + { id: "export-logs", title: "Export your logs", description: "Set up the Fly Log Shipper to aggregate your app’s logs to a service of your choice. Read more about [Exporting logs](/docs/monitoring/exporting-logs/)."} + ], + c: params[:c] || "", + o: params[:o] || "", + h: params[:h] || "" +) %> + +## CI/CD + +<%= render ChecklistComponent.new( + items: [ + { id: "review-apps", title: "Generate review apps with GitHub Actions", description: "Automatically generate ephemeral review apps on Fly.io for each pull request (PR) using GitHub Actions. Learn more about [Git Branch Preview Environments on GitHub](/docs/blueprints/review-apps-guide/)."}, + { id: "deploy-with-github-actions", title: "Deploy with GitHub Actions", description: "Set up your app for continuous deployment to Fly.io from the app’s GitHub repository. Find out how to use [Continuous Deployment with Fly.io and GitHub Actions](/docs/app-guides/continuous-deployment-with-github-actions/)."} + ], + c: params[:c] || "", + o: params[:o] || "", + h: params[:h] || "" +) %> + +## Get support + +<%= render ChecklistComponent.new( + items: [ + { id: "community", title: "Get answers in our community forum", description: "Check out our [community forum](https://community.fly.io/) to talk about your project and get help."}, + { id: "email-support", title: "Consider a purchasing a support plan", description: "Standard, Premium, or Enterprise support packages are available to purchase. Learn more about [Support plans](https://fly.io/support)."} + ], + c: params[:c] || "", + o: params[:o] || "", + h: params[:h] || "" +) %> diff --git a/apps/index.html.markerb b/apps/index.html.markerb index 2c315efe91..e6e43834bc 100644 --- a/apps/index.html.markerb +++ b/apps/index.html.markerb @@ -1,52 +1,51 @@ --- -title: "Fly Launch" +title: "Apps on Fly.io" layout: docs toc: false -nav: firecracker +nav: apps --- -Fly Launch is a collection of opinionated Fly.io platform features that help you configure and orchestrate your app as a unit. The following are some Fly Launch features: - -- the `fly launch` command to get started -- the `fly.toml` file for app configuration -- the `fly deploy` command to deploy all your app's Machines -- the `fly scale` command for horizontal and vertical scaling - -You can also configure and manage Fly Machines individually; see the [Fly Machines](/docs/machines/) docs. - -Learn how to use Fly Launch: - -
    -
  • - <%= nav_link "Launch a New Fly App", "/docs/apps/launch/" %> -
    • -
  • - <%= nav_link "Deploy a Fly App", "/docs/apps/deploy/" %> -
    • -
  • - <%= nav_link "Get Information about an App", "/docs/apps/info/" %> -
    • -
  • - <%= nav_link "Add Volume Storage", "/docs/apps/volume-storage/" %> -
    • -
  • - <%= nav_link "Scale Machine CPU and RAM", "/docs/apps/scale-machine/" %> -
    • -
  • - <%= nav_link "Add or Remove Machines", "/docs/apps/scale-count/" %> -
    • -
  • - <%= nav_link "Automatically Stop and Start Machines", "/docs/apps/autostart-stop/" %> -
  • - <%= nav_link "Restart an App", "/docs/apps/restart/" %> -
    • -
  • - <%= nav_link "Run Multiple Processes in an App", "/docs/apps/processes/" %> -
    • -
  • - <%= nav_link "Delete an App", "/docs/apps/delete/" %> -
    • -
  • - <%= nav_link "Fly Launch Configuration Reference", "/docs/reference/configuration/" %> -
    • -
+
+ Illustration by Annie Ruygt of a green block with A written on it, followed by a database stack in a dancing move +
+ +An app on Fly.io can be anything from a simple frontend web app to a complex arrangement of processes and Machines all doing their own thing. Check out the [Fly Apps overview](/docs/apps/overview/) for details. + + +## Fly Launch + +_If you're looking to deploy an app on Fly.io, you probably want Fly Launch. Use Fly Launch to create your app and then manage the whole lifecycle, from starting to scaling to changing and redeploying._ + +- **[Get started](/docs/getting-started/):** Launch your own app or try a demo app. + +- **[Use Fly Launch](/docs/launch/):** Learn how to create, configure, deploy, and scale your app. + +--- + +## Secrets + +_Store sensitive data, like usernames and passwords, as secrets in your app. Secrets are made available to the app as environment variables._ + +- [Set app secrets to be available at runtime](/docs/apps/secrets/) +- [Mount secrets in your Dockerfile to be available at buildtime](/docs/apps/build-secrets/) + +--- + +## Production apps + +_Get your app ready for production on Fly.io._ + +- [Going to production checklist](/docs/apps/going-to-production/) +- [App availability and resiliency](/docs/apps/app-availability/) +- [Fine-tune your app](/docs/apps/fine-tune-apps/) +- [Guidelines for concurrency settings](/docs/apps/concurrency/) + +--- + +## Related topics + +- [Databases & Storage](/docs/database-storage-guides/) +- [Networking](/docs/networking/) +- [Monitoring](/docs/monitoring/) +- [Security](/docs/security) +- [Blueprints: Fly.io patterns for your projects](/docs/blueprints/) diff --git a/apps/info.html.md b/apps/info.html.md index ebf50a5b63..a17e5cf3a7 100644 --- a/apps/info.html.md +++ b/apps/info.html.md @@ -1,14 +1,16 @@ --- -title: Get Information about an App -objective: +title: Get information about an app layout: docs -nav: firecracker -order: 15 +nav: apps --- -Once your Fly App is launched, `flyctl` has various tools for getting information about it. You can also find a lot of information on your Fly.io [web dashboard](https://fly.io/dashboard). +
+ +
-## Find all your Apps +Once your Fly App is launched, `flyctl` has various tools for getting information about it. You can also find a lot of information on your Fly.io [dashboard](https://fly.io/dashboard). + +## Find all your apps You can see a list of all your Fly Apps: @@ -21,7 +23,7 @@ testrun personal deployed machines 21h17m ago my-app personal suspended machines 2023-11-15T23:33:07Z ``` -## App Overviews +## App overviews If you want a brief app overview, including a list of Machines on that app with their current status, use `fly status`: @@ -41,7 +43,7 @@ ID STATE REGION HEALTH CHECKS IMAGE As with many flyctl commands, if you leave off the `-a` flag, `fly status` will infer the app name from the `fly.toml` file in the working directory, if there is one. -`fly machine list` yields a different set of information for each Machine, including the Machine's [internal IPv6 address](/docs/reference/private-networking/): +`fly machine list` yields a different set of information for each Machine, including the Machine's [internal IPv6 address](/docs/networking/private-networking/): ```cmd fly machine list -a testrun @@ -104,18 +106,19 @@ TCP 80 => 8080 [HTTP] True ## Public IP addresses -Find your app's public Anycast IPs with `fly ips list`. +Find your app's public Anycast IPs and private Flycast IPs with `fly ips list`. ```cmd fly ips list ``` ```out -VERSION IP TYPE REGION CREATED AT -v6 2a09:8280:1::d285 public global 2023-01-25T21:35:29Z -v4 66.241.125.211 public (shared) +VERSION IP TYPE REGION CREATED AT +v6 2a09:8280:1::2d:678b public (dedicated) global Sep 1 2023 19:47 +v6 fdaa:2:45b:0:1::23 private global Mar 16 2024 18:20 +v4 66.241.124.63 public (shared) Jan 1 0001 00:00 ``` -Read more about [Public Network Services](/docs/reference/services/) and [Private Networking](/docs/reference/private-networking/). +Read more about [Public networking](/docs/networking/services/) and [Private networking](/docs/networking/private-networking/). ## Check on it from inside @@ -146,14 +149,13 @@ tmpfs 113224 0 113224 0% /sys/fs/cgroup /dev/vdb 1011672 2564 940500 1% /storage ``` -
If you are running an interactive command (like a shell, IEx, or the Rails console), you may need to use the `--pty` flag. This tells the SSH server to run the command in a pseudo-terminal. (If you're familiar with OpenSSH, this is like the `-t` flag for `ssh`.)
+
If you are running an interactive command (like a shell, IEx, Django management command, or the Rails console), you may need to use the `--pty` flag. This tells the SSH server to run the command in a pseudo-terminal. (If you're familiar with OpenSSH, this is like the `-t` flag for `ssh`.)
-## Inspect the Current Configuration of a Deployed App or Machine +## Inspect the current configuration of a deployed app or Machine Machines can be configured individually, but Fly Launch applies the app's config on `fly deploy` to all Machines that are administered by the app. Display the app configuration in JSON format with `fly config show`. - ### Show the app config ```cmd @@ -269,4 +271,4 @@ fly logs -a testrun `fly logs` stays open, watching the logs, until you stop it (ctrl-C). -You can also [ship logs to an external service](/blog/shipping-logs/). +Learn more about [logging on Fly.io](/docs/monitoring/logging-overview/). diff --git a/apps/launch.html.markerb b/apps/launch.html.markerb deleted file mode 100644 index a206dde94f..0000000000 --- a/apps/launch.html.markerb +++ /dev/null @@ -1,92 +0,0 @@ ---- -title: Launch a New App on Fly.io -objective: -layout: docs -nav: firecracker -order: 10 -redirect_from: /launch/ ---- - -Fly Launch helps you configure and orchestrate your app. - -To create a brand new app on Fly.io, run this command from the source directory of your project: - -```cmd -fly launch -``` - -If you're in a hurry to try your first launch, we have a condensed [Hands-on](/docs/hands-on/) that launches a super-simple demo app. Or check out our [language- and framework-specific launch guides](/docs/languages-and-frameworks/). - -## Ingredients for a successful `fly launch` - -Here are the components of a successful launch, ready for the first deployment: - -* A way to get a Docker image that we can use to get your app running on a Machine. -* A `fly.toml` file, created by `fly launch` or provided by you, for your app's Fly.io-specific [configuration](/docs/reference/configuration/). -* Optional platform resources such as [public IP addresses](/docs/reference/services/), [Fly Volumes](/docs/reference/volumes), [app secrets](/docs/reference/secrets/), [Fly Postgres](/docs/postgres/) clusters, and integrated resources like [Upstash for Redis](/docs/reference/redis/). - -## Framework launch scanners - -Depending on your project, `fly launch` may be able to look at your app's source code and get through that [ingredient list](#ingredients-for-a-successful-fly-launch), straight to a ready-to-deploy Fly App. This is most likely to work for frameworks on which Fly.io has people specializing full time. Right now that's [Elixir/Phoenix](/docs/elixir/), [Laravel](/docs/laravel/), [Rails](/docs/rails/), and [Django](/docs/django/), but we also have launch guides for other [languages and frameworks](/docs/languages-and-frameworks/). - -Our best scanners furnish a Dockerfile from which your app's image will be built. Some of our terrible older scanners may invoke [buildpacks](/docs/reference/builders/#buildpacks), which tend to be slow and brittle. - -Running `fly launch` in a directory containing a working Django app (it happens to be the one from [our Django getting-started example](/docs/django/getting-started/)): - -```cmd -fly launch -``` -```out -Scanning source code -Detected a Django app -Creating app in /flyio/hello-django -We're about to launch your Django app on Fly.io. Here's what you're getting: - -Organization: MyName (fly launch defaults to the personal org) -Name: hello-django (derived from your directory name) -Region: Amsterdam, Netherlands (this is the fastest region for you) -App Machines: shared-cpu-1x, 1GB RAM (most apps need about 1GB of RAM) -Postgres: (not requested) -Redis: (not requested) - -? Do you want to tweak these settings before proceeding? Yes -... -``` - -The flyctl Django scanner has taken ownership of the launch. You can tweak the basic settings on the Fly Launch web page and then run 'fly deploy' to deploy the new app. Visit our [Django guide](/docs/django/getting-started/) to see how that story will end. (Spoiler: it has a happy ending.) - -## Custom launch - -You can nudge `fly launch` to better suit your project. - -### Point to an image or use a Dockerfile to build - -Tell `fly launch` how you want to get the Docker image for your app, using either the `--image` or `--dockerfile` option, or by catching the Dockerfile launch scanner's attention with the presence of a [Dockerfile](/docs/languages-and-frameworks/dockerfile/) in your project source directory. The Dockerfile scanner doesn't do a lot of configuration, but it prevents other scanners from taking over. - -The actual Docker image build (or image pull) for a Fly App takes place during deployment. `fly launch` sets the stage by recording how to build, or get, the image, and both the first and all later deploys use that information. - -### Customize the configuration file - -You can provide your own `fly.toml` and `fly launch` will offer to copy that configuration to a new app. `fly.toml` sets a starting point for the app configuration, and in some cases a framework launch scanner might overwrite parts of it. - -There are also [a number of other options](/docs/flyctl/launch/) you can use to exert control over `fly launch`. - -If `fly launch` doesn't have a scanner that can set up your app automatically, it will still initialize a new Fly App in your Fly.io organization and provide you with a default app configuration that's a reasonable starting point for a simple web app. - -You can also perform an entirely manual "launch", skipping all the launch scanners and full-service resource provisioning, using `fly apps create`, a hand-crafted (or copied) `fly.toml`, and step-by-step resource provisioning, followed by `fly deploy`. - -## After `fly launch` - -If you've run `fly launch` but haven't deployed yet (hint: you can do this with `fly launch --no-deploy`), or you deployed but want to make changes, then you can: - -* change your configuration -* update your app source -* change or provision platform resources such as [public IP addresses](/docs/reference/services/), [Fly Volumes](/docs/reference/volumes), [app secrets](/docs/reference/secrets/), [Fly Postgres](/docs/postgres/) clusters, or integrated resources like [Upstash for Redis](/docs/reference/redis/) - -And then deploy (or redeploy) with [`fly deploy`](/docs/apps/deploy/). - -After you deploy your app: - -* Check up on it using one or more of the techniques in [Get Information About an App](/docs/apps/info/). -* Get ready to [go to production](/docs/going-to-production/), read up on [app availability and resiliency features](/docs/reference/app-availability/), and learn how to [scale the number of Machines](/docs/apps/scale-count/) or [scale Machine CPU and RAM](/docs/apps/scale-machine/). -* Go deeper and do even more, at a lower level, with [Machines](/docs/machines/). diff --git a/apps/move-app-org.html.markerb b/apps/move-app-org.html.markerb new file mode 100644 index 0000000000..0fa8635194 --- /dev/null +++ b/apps/move-app-org.html.markerb @@ -0,0 +1,48 @@ +--- +title: Move an app between organizations +layout: docs +nav: apps +--- + +
+ Illustration by Annie Ruygt of a figure jumping from one mountian to another +
+ +You might want to move an app to a new org to hand off your completed work to a client, to take advantage of a new pricing model, or to move a production app out of your personal organization. + +You can move an entire app with its resources to another organization with the `fly apps move` command. You'll need to be a member of both organizations to move an app. + +To move an app from one organization to another: + +```cmd +fly apps move --org +``` + +
+**Fly Postgres:** You can't use the `fly apps move` command for Fly Postgres apps. To move a Postgres app to another organization, create a new Postgres app under the target organization, and then [restore the data from your current Postgres app volume snapshot](/docs/postgres/managing/backup-and-restore/#restoring-from-a-snapshot). +
+ +## App downtime + +Your app will have up to a few minutes of downtime while the move operation completes. + +## Resources moved with the app + +The following app resources are transferred to the new org automatically: + +- Fly Machines and Fly Volumes (including data) +- environment variables and secrets +- certificates and domain names +- LiteFS databases (when `$FLY_APP_NAME` is used for your Consul key in `litefs.yml`) + +## Resources that you need to manually reconfigure + +The following extension services need to be reconfigured for the app after the move: + +- **[Upstash for Redis](/docs/upstash/redis/):** Upstash Redis is only available over an organization's private network. After you move the app, you'll need to provision a new database for the new org. +- **[Tigris object storage](/docs/tigris/):** Before moving the app, you'll need to delete the bucket, then recreate it in the new org and set the new secrets. Billing for the new bucket will be tied to the new org. + + +## Read more + +[App handover considerations and approaches](/docs/apps/app-handover-guide/) \ No newline at end of file diff --git a/apps/overview.html.markerb b/apps/overview.html.markerb new file mode 100644 index 0000000000..b38c0b340f --- /dev/null +++ b/apps/overview.html.markerb @@ -0,0 +1,62 @@ +--- +title: "Fly Apps overview" +layout: docs +toc: false +nav: apps +redirect_from: /docs/reference/apps/ +--- + +A Fly App is an abstraction for a group of Fly Machines running your code on Fly.io. You can create and manage your app as a whole using [Fly Launch](/docs/launch/), but you can also have a Fly App with individual Machines running tasks or user code. + +For a quick primer on Fly Launch, Fly Machines, and Fly Apps, see [Fly.io essentials](/docs/getting-started/essentials/). + +## A Fly App can be almost anything + +From an admin point of view, a Fly App is just a group of Machines (with optional attached volumes) that belongs to one organization. + +From a developer point of view, a Fly App might be: + +* a fullstack application (or just part of one) +* a database +* a few Machines running tasks, or a bunch of Machines, all with different configs, doing things you want them to do +* anything you can think of doing with fast-launching Machines, including [GPU Machines](/docs/gpus/) for AI/ML workloads + +All the apps in your organization can communicate over a [private network](/docs/networking/private-networking/), so it’s also possible to have multiple apps working together as one system. + +## Components of a Fly App + +Fly Apps include: + +* app configuration +* provisioned resources +* Anycast IP addresses +* certificates +* custom domains +* secrets +* Fly Volumes (optional) + +Fly Apps run on flyd. Learn more about how flyd works and how it came to be in the blog post: [Carving the Scheduler Out of Our Orchestrator](https://fly.io/blog/carving-the-scheduler-out-of-our-orchestrator/). + +## Ways to create an app + +There might be an infinite number of potential apps you can run on Fly.io, but there are only a few ways to create one. + +### Create and deploy with Fly Launch + +Fly Launch is perfect for most apps and databases. Use Fly Launch to create your app and then manage the whole lifecycle, from starting to scaling to changing and redeploying. [Get started](/docs/getting-started/) with Fly Launch or learn more about using [Fly Launch](/docs/launch/) to create, configure, deploy, and scale your app. + +### Create an app manually + +Most of the time, all you need is [Fly Launch](/docs/launch/) to create and manage your app. But you can skip all the handy launch scanners and resource provisioning Fly Launch offers and use the `fly apps create` command or the Machines API to create an app and then piece it together. + +You can get a [`fly.toml` config file](/docs/reference/configuration/) from an example app or hand-craft one yourself. Or if you don't need app-wide config, you can use the Fly App to hold the Machines you plan to spin up individually based on a particular image. + +Using the `fly apps create` command is useful when deploying our [autoscaler app](/docs/launch/autoscale-by-metric/), or example apps created by others, that already have `fly.toml` and Dockerfiles ready to go. + +Creating apps with the Machines API might be the right choice if you want to create apps for your own users with pre-defined or custom `fly.toml` files. + +For more information, see the [`fly apps create` docs](/docs/flyctl/apps-create/) and the [create App resource docs](/docs/machines/api/apps-resource/#create-a-fly-app). + +
+**Tip:** When you [create a Machine with `fly machine run`](/docs/machines/flyctl/fly-machine-run/) without specifying an app to put it in, you automatically get a new app at the same time. This scenario might be useful when you're spinning up individual Machines on-demand for multiple users or tasks. +
diff --git a/apps/restart.html.markerb b/apps/restart.html.markerb index f4841c89c0..6a5dc3a636 100644 --- a/apps/restart.html.markerb +++ b/apps/restart.html.markerb @@ -1,25 +1,27 @@ --- -title: Restart an App or a Machine -objective: +title: Restart apps or Machines layout: docs -nav: firecracker -order: 70 +nav: apps --- -Running `fly deploy` on an App creates a new release of the App, and restarts the Machines that it manages. +
+ +
-Sometimes you don't want to update anything; you just want to reboot and start the root file system afresh. (Yes, restarting wipes the ephemeral file system, just as a `fly deploy` or `fly machine update` will.) +Running `fly deploy` on an app creates a new release of the app, and updates and restarts the Machines that belong to Fly Launch. -## Restart every Machine in the App +Sometimes you don't want to update anything; you just want to reboot and start the root file system afresh. Restarting wipes the ephemeral file system, just as a `fly deploy` or `fly machine update` will. -`fly apps restart ` restarts all Machines in the App— Machines that belong to Fly Launch as well as any other standalone Machines you might have created within the App. +## Restart every Machine managed by Fly Launch -## Restart an Individual Machine +`fly apps restart ` restarts all Machines in the app that are managed by Fly Launch. In other words, all the Machines deployed with `fly deploy` and configured with `fly.toml`. -If one particular Machine needs a kick, restart it using `fly machine restart`: +## Restart individual Machines + +If one particular Machine needs a kick—or you want to restart an unmanaged Machine—then run `fly machine restart`: ```cmd -fly machine restart +fly machine restart ``` -You can provide multiple ``s to restart several Machines in one command. +You can provide multiple ``s to restart several Machines in one command. diff --git a/apps/scale-machine.html.markerb b/apps/scale-machine.html.markerb deleted file mode 100644 index bdff051862..0000000000 --- a/apps/scale-machine.html.markerb +++ /dev/null @@ -1,210 +0,0 @@ ---- -title: Scale Machine CPU and RAM -objective: -layout: docs -nav: firecracker -order: 50 ---- - -[**`fly scale`**](/docs/flyctl/scale) commands apply VM memory and CPU settings to entire process groups in a Fly App. There are two subcommands for scaling these per-VM resources: `fly scale vm` applies a preset CPU/RAM combination; `fly scale memory` adjusts RAM relative to the preset's base RAM. - -You can scale an app even if it has crashed. Its VMs are restarted with the new specification. - -## Check the VM resources on an app -Here's a simple web app with two Machines running in different regions: two in Toronto and one in Tokyo. All the app's Machines belong to the default process group, `app`, since I didn't explicitly configure any [processes](/docs/apps/processes/). - -```cmd -fly status -``` -```out -App - Name = testrun - Owner = personal - Hostname = testrun.fly.dev - Image = testrun:deployment-01GWZY7ZVJ2HNED4B0KZBPS3AQ - Platform = machines - -Machines -ID PROCESS VERSION REGION STATE HEALTH CHECKS LAST UPDATED -148ed599c14189 app 3 yyz started 1 total, 1 passing 2023-04-04T19:30:57Z -32874400f35285 app 3 yyz started 1 total, 1 passing 2023-04-04T19:33:44Z -9080e6e1f94987 app 3 nrt started 1 total, 1 passing 2023-04-04T19:31:22Z -``` - -`fly scale show` shows the CPU and RAM settings for all the Machines deployed using `fly deploy` under this app. - -```cmd -fly scale show -``` -```out -VM Resources for app: testrun - -Groups -NAME COUNT KIND CPUS MEMORY REGIONS -app 3 shared 1 256 MB nrt,yyz(2) -``` - -These Machines are running at the `shared-cpu-1x` preset scale, with a single shared vCPU and 256MB RAM. - -## Select a preset CPU/RAM combination - -There are a number of VM size presets available. See the list of valid named presets with `fly platform vm-sizes`. - -Scale to a different preset using `fly scale vm`. In general, you should choose a named VM "size" based on your desired CPU type and scale; RAM can be increased separately. - -```cmd -fly scale vm shared-cpu-2x -``` -```out -Updating machine 148ed599c14189 - Waiting for 148ed599c14189 to become healthy (started, 1/1) -Machine 148ed599c14189 updated successfully! -Updating machine 32874400f35285 - Waiting for 32874400f35285 to become healthy (started, 1/1) -Machine 32874400f35285 updated successfully! -Updating machine 9080e6e1f94987 - Waiting for 9080e6e1f94987 to become healthy (started, 1/1) -Machine 9080e6e1f94987 updated successfully! -Scaled VM Type to 'shared-cpu-2x' - CPU Cores: 2 - Memory: 512 MB -``` -Check that the `app` process group has had this scale applied: - -```cmd -fly scale show -``` -```out -VM Resources for app: testrun - -Groups -NAME COUNT KIND CPUS MEMORY REGIONS -app 3 shared 2 512 MB nrt,yyz(2) -``` - -You can also confirm that an individual Machine's config matches this, using `fly machine status`: - -```cmd -fly machine status 148ed599c14189 -``` -```out -Machine ID: 148ed599c14189 -Instance ID: 01GX6Q2WE04M85XTHGPYGJK4X6 -State: started - -VM - ... - Process Group = app - CPU Kind = shared - vCPUs = 2 - Memory = 512 - ... -``` - -Looks good! - -## Add RAM - -If you are happy with the provisioned CPU resources, but want more memory, then use `fly scale memory` to top up the RAM. - -If your app crashes with an out-of-memory error, then scale up its RAM. Flyctl restarts the Machines to use the new setting. - -```cmd -fly scale memory 4096 -``` -```out -Updating machine 32874400f35285 - Waiting for 32874400f35285 to become healthy (started, 1/1) -Machine 32874400f35285 updated successfully! -Updating machine 148ed599c14189 - Waiting for 148ed599c14189 to become healthy (started, 1/1) -Machine 148ed599c14189 updated successfully! -Updating machine 9080e6e1f94987 - Waiting for 9080e6e1f94987 to become healthy (started, 1/1) -Machine 9080e6e1f94987 updated successfully! -Scaled VM Type to 'shared-cpu-2x' - CPU Cores: 2 - Memory: 4096 MB -``` - -```cmd -fly scale show -``` -```out -VM Resources for app: testrun - -Groups -NAME COUNT KIND CPUS MEMORY REGIONS -app 3 shared 2 4096 MB nrt,yyz(2) -``` - - -If you try to set an incompatible CPU/RAM combination through `fly scale memory`, flyctl will let you know. There's a list of allowed CPU/RAM combinations and their prices on [our Pricing page](/docs/about/pricing/). - - -## Scale by process group - -Use the `--process-group` option to specify the process group to scale, with either `fly scale vm` or `fly scale memory`. - -
-**Note**: The `--process-group` option is aliased to `-g` for faster command entry. -
- -Here's an app with two process groups defined: - -```cmd -fly scale show -``` -```out -VM Resources for app: mr18-2 - -Groups -NAME COUNT KIND CPUS MEMORY REGIONS -worker 2 shared 1 512 MB ams,yyz -web 1 shared 1 512 MB yyz -``` - -Say the workers are constantly crunching data and need to be bigger. You can scale a single process group using the `--process-group` option: - -```cmd -fly scale vm performance-2x --process-group worker -``` -```out -Updating machine 0e286561f35586 -No health checks found -Machine 0e286561f35586 updated successfully! -Updating machine 32873d9b012048 -No health checks found -Machine 32873d9b012048 updated successfully! -Scaled VM Type for 'worker' to 'performance-2x' - CPU Cores: 2 - Memory: 4096 MB -``` - -Check the result: - -```cmd -fly scale show -``` -```out -VM Resources for app: mr18-2 - -Groups -NAME COUNT KIND CPUS MEMORY REGIONS -worker 2 performance 2 4096 MB ams,yyz -web 1 shared 1 512 MB yyz -``` - -## Machines not belonging to Fly Launch - -If an app has Machines not belonging to Fly Launch—that is, created using `fly machine run` or the Machines API, or `fly machine clone`d from such a Machine—`fly status` will warn you of their existence. - -The app-wide `fly scale` commands do not apply to these Machines, but you can scale any Machine individually with `fly machine update`: - -``` -fly machine update --vm-size shared-cpu-2x 21781973f03e89 -fly machine update --vm-memory 1024 21781973f03e89 -fly machine update --vm-cpus 2 21781973f03e89 -``` - -Again, if you try to set an incompatible CPU/RAM combination through `fly machine update --vm-memory` or `fly machine update --vm-cpus`, flyctl will let you know. diff --git a/apps/secrets.html.markerb b/apps/secrets.html.markerb new file mode 100644 index 0000000000..16a46d2b5c --- /dev/null +++ b/apps/secrets.html.markerb @@ -0,0 +1,100 @@ +--- +title: Secrets and Fly Apps +layout: docs +nav: apps +redirect_from: /docs/reference/secrets/ +--- + +## Overview + +You can specify secrets for your Fly App using the `fly secrets` command. Secrets allow sensitive values, such as credentials, to be passed securely to your Fly App. The secret is encrypted and stored in a vault. An app's secrets are available as environment variables at runtime on every Machine belonging to that Fly App, whether the Machine is managed by Fly Launch or not. + +Data stored as secrets doesn't have to be sensitive; secrets are made available to the app as environment variables to use for whatever purpose you like. + +If you need secrets to be available when building your Docker image, see [Build secrets](/docs/apps/build-secrets/). + +## Architecture + +Secrets are stored in an encrypted vault. When you set a secret through `flyctl`, it sends the secret value through our API, which writes to the vault for your specific Fly App. The API servers can only encrypt; they cannot decrypt secret values. Secret values are never logged. + +When we launch a Machine for your app, we issue a temporary auth token to the host it runs on. The Fly.io agent on the host uses this token to decrypt your app secrets and inject them into your Machine as environment variables at boot time. When you destroy your Machines, the host environment no longer has access to your app secrets. + +
+**Warning:** `flyctl` and our API servers are designed to prevent user secrets from being extracted. However, secrets are available to your application code as environment variables. People with deploy access **can** deploy code that reads secret values and prints them to logs, or writes them to unencrypted data stores. +
+ +## Working with Secrets +### Set secrets + +The `fly secrets set` command sets one or more app secrets, then updates each Machine belonging to that Fly App. This involves a restart of the Machine and a consequent reset of its ephemeral file system. + +The following example sets a secret that's available as the `DATABASE_URL` environment variable within your application processes: + +<%= partial "docs/partials/set_secrets" %> + +To set, or update, a secret in the app's vault, but defer updating the Machines to later, use the `--stage` option. For example: + +```cmd +fly secrets set DATABASE_URL=postgres://example.com/mydb --stage +``` + +In the preceding example, the staged secret will be available only on Machines that are started or updated after the `fly secrets set` command was run. + +The `secrets` command can also take secrets from `stdin`. For commands and options, see the [`fly secrets` docs](/docs/flyctl/secrets/) or run `fly secrets --help`. + +
+**Note:** You can update a machine by triggering a new release with `fly deploy`. Alternatively, the `fly secrets deploy` command will redeploy the current release with the staged secrets. This is helpful if you want to skip rebuilding the image from source code.
+ +### List secrets + +List secrets that are set for your app. The list shows only the secret name; the value is not shown as it is a secret. + +For example: + +```cmd +fly secrets list +``` +```output + NAME | DIGEST | DATE ++--------------+----------------------------------+---------+ + MY_SECRET | b9e37b7b239ee4aefc75352fe3fa6dc6 | 17s ago + DATABASE_URL | cdbe3268a82bfe993921b9cae2a526af | 17s ago +``` + +For security reasons, we do not allow read access to the plain-text values of secrets. + +### Remove secrets + +Remove one or more secret values from your app by name. + +The following example removes two secrets from the app: + +```cmd +fly secrets unset MY_SECRET DATABASE_URL +``` + +### Mounting Secrets as Files + +You can write a secret’s value directly to the machine's filesystem at startup using the `[[files]]` section in `fly.toml`. This is useful for things like private keys your app expects on disk. + +Secrets must be base64‑encoded. + +Example: + +```cmd +fly secrets set SUPER_SECRET=$(cat filename.txt | base64) +``` + +`secret_name`: The name of an app secret that’s been set with `fly secrets set`. The value of the secret will be written to the file. The referenced secret’s value must be base64 encoded. + +`guest_path`: The full path inside the Machine where the file will be written at boot. + +```toml +[[files]] + guest_path = "/path/to/secret.txt" + secret_name = "SUPER_SECRET" +``` + +You can use this with other `[[files]]` entries, check the `fly.toml` [reference](/docs/reference/configuration/#the-files-section) for more details. + + diff --git a/apps/trouble-host-unavailable.html.markerb b/apps/trouble-host-unavailable.html.markerb index a539f9425d..451d4b8b41 100644 --- a/apps/trouble-host-unavailable.html.markerb +++ b/apps/trouble-host-unavailable.html.markerb @@ -2,9 +2,13 @@ title: Troubleshoot apps when a host is unavailable objective: layout: docs -nav: firecracker +nav: apps --- +
+ +
+ Fly.io has multiple hosts (physical hardware) in every supported [region](/docs/reference/regions/). From time to time, a hardware failure, connectivity issues, or other factors will make a host unavailable. A single-host issue may cause a sustained or intermittent outage, or generally degraded performance, and this will disproportionately affect apps that run on a single Machine. A host can be down for an hour, or a day, or sometimes longer. @@ -40,7 +44,7 @@ For apps without volumes, you can bring up a new Machine by scaling to zero and ```
-**Note:** When you scale all your app's Machines to zero, the Machine size resets to shared-cpu-1x with 256MB of RAM. Use one of the `--vm-` options to size the Machines. For example: `fly scale count 2 --vm-size shared-cpu-4x`. See the [fly scale count docs](docs/flyctl/scale-count/) for options. +**Note:** When you scale all your app's Machines to zero, the Machine size resets to shared-cpu-1x with 256MB of RAM. Use one of the `--vm-` options to size the Machines. For example: `fly scale count 2 --vm-size shared-cpu-4x`. See the [fly scale count docs](/docs/flyctl/scale-count/) for options.
@@ -62,7 +66,7 @@ Volumes are pinned to physical hosts, so when there's a host outage the volume i 1. Run `fly volumes list` and copy the volume ID. -1. Run `fly volumes snapshots ` to list the volume’s snapshots and then copy the snapshot ID. +1. Run `fly volumes snapshots list ` to list the volume’s snapshots and then copy the snapshot ID. 1. Run `fly volumes create --snapshot-id -s ` to create a new volume from a stored snapshot. @@ -72,7 +76,7 @@ Volumes are pinned to physical hosts, so when there's a host outage the volume i If you're running a high availability Postgres cluster with multiple nodes, a host issue impacting one of your nodes shouldn't cause an issue; by default we run each node on a separate host. If the host your primary node is on goes down, the cluster will fail over to a healthy node. -However, if your database is running on a single Machine, and you don’t have any replicas to fail over to, then you won’t be able to connect during the host outage. Similar to the single volume steps above, you can create a new Postgres app from your most recent volume snapshot using `fly postgres create --snapshot-id `. See [Backup, restores, & snapshots](/docs/postgres/managing/backup-and-restore/) for details. +However, if your database is running on a single Machine, and you don’t have any replicas to fail over to, then you won’t be able to connect during the host outage. Similar to the single volume steps above, you can create a new Postgres app from your most recent volume snapshot using `fly postgres create --snapshot-id --image-ref `. See [Backup, restores, & snapshots](/docs/postgres/managing/backup-and-restore/) for details. ## Prevent downtime when there's a single host issue @@ -80,4 +84,7 @@ Fly.io strongly recommends running at least two Machines per app in your primary ## Related topics +- [App availability and resiliency](/docs/reference/app-availability/) +- [Resilient apps use multiple Machines](/docs/blueprints/resilient-apps-multiple-machines/) - [Troubleshooting your deployment](/docs/getting-started/troubleshooting/) +- [Fly.io error codes and troubleshooting](/docs/monitoring/error-codes/) diff --git a/apps/volume-manage.html.markerb b/apps/volume-manage.html.markerb deleted file mode 100644 index 569b7d00f2..0000000000 --- a/apps/volume-manage.html.markerb +++ /dev/null @@ -1,199 +0,0 @@ ---- -title: Manage volume storage -layout: docs -nav: firecracker -order: 40 ---- - -Manage Fly Volumes using the [`fly volumes`](/docs/flyctl/volumes/) command. - -Fly Volumes are local persistent storage for [Fly Machines](/docs/machines/). Learn [how Fly Volumes work](/docs/reference/volumes/). - -
**Note**: `fly volumes` is aliased to `fly volume` and `fly vol`.
- -## Create a volume - -Run: - -```cmd -fly vol create -``` - -Use the `-r` option to set a [region](/docs/reference/regions/), or select a region when prompted. - -For options, refer to the [`fly volumes create` docs](/docs/flyctl/volumes-create/) or run `fly vol create -h`. - -To add a volume to your app, see [Add volume storage](/docs/apps/volume-storage/). - -## Access a volume - -You can access and write to a volume on a Machine just like a regular directory. - -For Machines managed with Fly Launch, the `destination` under `[mounts]` in `fly.toml` is the path for the mounted volumes. - -For Machines managed individually, specify the mount path in the `fly machine clone` command when you [clone a Machine and add a volume](/docs/apps/volume-storage/#add-a-volume-to-a-machine-clone). - -## Extend a volume - -You can extend (increase) a volume's size, but you can't make a volume smaller. - -1. Run `fly volumes list` and copy the ID of the volume to extend. - -1. Extend the volume size: - - ```cmd - `fly vol extend -s ` - ``` - -1. (Optional) Check the new volume size in the Machine's file system: - - ```cmd - fly ssh console -s -C df - ``` - - Example output: - ```out - ? Select VM: [Use arrows to move, type to filter] - > yyz: 4d891de2f66587 fdaa:0:3b99:a7b:ef:8cc4:dc49:2 withered-shadow-4027 - Connecting to fdaa:0:3b99:a7b:ef:8cc4:dc49:2... complete - Filesystem 1K-blocks Used Available Use% Mounted on - devtmpfs 103068 0 103068 0% /dev - /dev/vda 8191416 172752 7582852 3% / - shm 113224 0 113224 0% /dev/shm - tmpfs 113224 0 113224 0% /sys/fs/cgroup - /dev/vdb 2043856 3072 1930400 1% /storage - ``` - - In the preceding example, the volume is mounted under `/storage` and has been resized to from 1GB to 2GB. The `df` command shows disk space in 1K blocks by default. Use the `-h` flag to return a more human-readable format. - -For options, refer to the [`fly volumes extend` docs](/docs/flyctl/volumes-extend/) or run `fly vol extend -h`. - -## Create a copy of a volume (Fork a volume) - -Create an exact copy of a volume, including its data. By default, we place the new volume on a separate physical host in the same region. - -
-**Important:** After you fork a volume, the new volume is independent of the source volume. The new volume and the source volume do not continue to sync. -
- -1. Run `fly volumes list` and copy the ID of the volume to fork. - -2. Fork the volume: - - ```cmd - fly volumes fork - ``` - - Example output: - ```out - ID: vol_grnoq5796wwj0dkv - Name: my_volume_name - App: my-app-name - Region: yyz - Zone: 511d - Size GB: 3 - Encrypted: true - Created at: 12 Oct 23 18:57 UTC - ``` - -3. (Optional) Attach the volume to a new Machine by cloning with `fly machine clone -r --attach-volume :` or [use `fly scale count`](/docs/apps/scale-count/#scale-an-app-with-volumes) to create a new Machine that picks up the unattached volume. - -For options, refer to the [`fly volumes fork` docs](/docs/flyctl/volumes-fork/) or run `fly vol fork -h`. - -## Restore a volume from a snapshot - -We take daily snapshots and keep them for five days. - -1. Run `fly volumes list` and copy ID of the volume to restore. - -1. List the volume's snapshots: - - ```cmd - fly vol snapshots - ``` - - Example output: - ```out - Snapshots - ID SIZE CREATED AT - vs_MgLAggLZkYx89fLy 17638389 1 hour ago - vs_1KRgwpDqZ2ll5tx 17649006 1 day ago - vs_nymJyYMwXpjxqTzJ 17677766 2 days ago - vs_R3OPAz5jBqzogF16 17689473 3 days ago - vs_pZlGZvq3gkAlAcaZ 17655830 4 days ago - vs_A9k6age3bQov6twj 17631880 5 days ago - ``` - -1. Restore from the volume snapshot into a new volume of equal or greater size: - - ```cmd - fly volumes create --snapshot-id -s - ``` - - Example output: - ```out - ? Select region: Sydney, Australia (syd) - ID: vol_mjn924o9l3q403lq - Name: pg_data - App: my-app-name - Region: syd - Zone: 180d - Size GB: 3 - Encrypted: true - Created at: 02 Aug 22 21:27 UTC - ``` - -For options, refer to the [`fly volumes snapshots` docs](/docs/flyctl/volumes-snapshots/) or run `fly vol snapshots -h`. - -## Clone a Machine with a volume - -Clone a Machine with a volume to create a new Machine in the same region with an empty volume. Use the `-r` option to clone the Machine into a different [region](/docs/reference/regions/). - -1. Run `fly status` and copy the Machine ID of the Machine to clone. - -1. Clone the Machine: - - ```cmd - fly m clone - ``` - -1. List volumes to check the result: - - ```cmd - fly volumes list - ``` - - Example output showing two volumes with attached Machines: - ```out - ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT - vol_ez1nvxkwl3jrmxl7 created data 1GB lhr 4de2 true 91851edb6ee983 39 seconds ago - vol_zmjnv8m81p5rywgx created data 1GB lhr b6a7 true 5683606c41098e 7 minutes ago - ``` - -
-Note: `fly machine clone` doesn't write data into the new volume. -
- -For options, refer to the [`fly machine clone` docs](/docs/flyctl/machine-clone/) or run `fly m clone -h`. - -## Destroy a volume - -
-Warning: When you destroy a volume, you permanently delete all its data. -
- -1. Run `fly volumes list` and copy ID of volume to destroy. - -2. Destroy the volume: - -```cmd -fly vol destroy -``` - -For options, refer to the [`fly vol destroy` docs](/docs/flyctl/volumes-destroy/) or run `fly vol destroy -h`. - -## Related topics - -- [Fly Volumes overview](/docs/reference/volumes/) -- [Add volume storage](/docs/apps/volume-storage/) -- [Scale an app with volumes](/docs/apps/scale-count/#scale-an-app-with-volumes) \ No newline at end of file diff --git a/apps/volume-storage.html.markerb b/apps/volume-storage.html.markerb deleted file mode 100644 index 4dbcdc416a..0000000000 --- a/apps/volume-storage.html.markerb +++ /dev/null @@ -1,187 +0,0 @@ ---- -title: Add volume storage -layout: docs -nav: firecracker -order: 30 ---- - -Fly Volumes are local persistent storage for [Fly Machines](/docs/machines/). Learn [how Fly Volumes work](/docs/reference/volumes/). - -## Launch a new app with a Fly Volume - -Use [Fly Launch](/docs/apps/) to create a new app with one Machine and an attached volume, and then clone the Machine to scale out. - -1. Launch a new app from your project source directory, specifying `--no-deploy` so it does not deploy immediately: - - ```cmd - fly launch --no-deploy - ``` - -1. After the app is created, add a [`[mounts]` section](/docs/reference/configuration/#the-mounts-section) in the app's `fly.toml`, where `source` is the volume name and `destination` is the directory where the volume should be mounted on the Machine file system. For example: - - ```toml - [mounts] - source="myapp_data" - destination="/data" - ``` - -1. Deploy the app: - - ```cmd - fly deploy - ``` - -1. [Confirm that the volume is attached to a Machine](#confirm-the-volume-is-attached-to-a-machine). - -1. (Recommended if your app handles replication) Clone the first Machine to scale out to two Machines with volumes: - - ```cmd - fly machine clone - ``` - - List volumes to check the result: - - ```cmd - fly volumes list - ``` - - Example output showing two volumes with attached Machines: - ```out - ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT - vol_ez1nvxkwl3jrmxl7 created data 1GB lhr 4de2 true 91851edb6ee983 39 seconds ago - vol_zmjnv8m81p5rywgx created data 1GB lhr b6a7 true 5683606c41098e 7 minutes ago - ``` - -
-Warning: `fly machine clone` doesn't write data into the new volume. -
- -## Add volumes to an existing app - -Add a volume to an app created with [Fly Launch](/docs/apps/). - -1. Add a [`[mounts]` section](/docs/reference/configuration/#the-mounts-section) in the app's `fly.toml`, where `source` is the volume name and `destination` is the directory where the volume should be mounted on the Machine file system. For example: - - ```toml - [mounts] - source="myapp_data" - destination="/data" - ``` - -1. Run `fly status` to check the [regions](/docs/reference/regions/) of the Machines and then create the volume in the same regions as your app's Machines. For example: - - ```cmd - fly volumes create -r - ``` - -1. Repeat step 2 for each Machine in the process group. If you create an app using the `fly launch` command, then the app will usually have two Machines in the `app` process by default. - -1. Deploy the app: - - ```cmd - fly deploy - ``` - -1. [Confirm that the volume is attached to a Machine](#confirm-the-volume-is-attached-to-a-machine). - -## Add a volume to an unmanaged Machine - -For Machines that aren't managed with Fly Launch (`fly.toml` and `fly deploy`), you can create a volume and attach it when you clone a Machine. You can also [clone a Machine with a volume](/docs/apps/volume-manage/#clone-a-machine-with-a-volume) to get a new Machine with an empty volume. - -1. Create the volume in the same region as your app. For example: - - ```cmd - fly volumes create -r - ``` - -1. Clone one of your app's Machines (with no volume) and attach the volume you just created: - - ```cmd - fly machine clone -r --attach-volume : - ``` - - `destination-mount-path` is the directory where the volume should be mounted on the file system. - - For example: - - ```cmd - fly machine clone 148eddeef09789 -r yyz --attach-volume vol_8l524yj0ko347zmp:/data - ``` - -1. Repeat the preceding steps as needed to create more Machines with volumes. - -1. [Confirm that the volume is attached to a Machine](#confirm-the-volume-is-attached-to-a-machine). - -1. (Optional) Destroy the Machine used to create the clone: - - ```cmd - fly machine destroy - ``` - -## Confirm the volume is attached to a Machine - -Use flyctl to check the status of volumes and Machines. - -### List the Machines - -List Machines to check attached volumes: - -```cmd -fly machine list -``` - -Example output: - -```out -1 machines have been retrieved from app my-app-name. -View them in the UI here - -my-app-name -ID NAME STATE REGION IMAGE IP ADDRESS VOLUME CREATED LAST UPDATED APP PLATFORM PROCESS GROUP SIZE -328773d3c47d85 my-app-name stopped yul flyio/myimageex:latest fdaa:2:45b:a7b:19c:bbd4:95bb:2 vol_6vjywx86ym8mq3xv 2023-08-20T23:09:24Z 2023-08-20T23:16:15Z v2 app shared-cpu-1x:256MB -``` - -### List the volumes - -List volumes to check attached Machines: - -```cmd -fly volumes list -``` - -Example output: - -```out -ID STATE NAME SIZE REGION ZONE ENCRYPTED ATTACHED VM CREATED AT -vol_zmjnv8m81p5rywgx created data 1GB lhr b6a7 true 5683606c41098e 3 minutes ago -``` - -### SSH into the Machine - -View the volume in the Machine file system: - -```cmd -fly ssh console -s -C df -``` - -Example output showing a 1GB volume mounted at `/data`: - -```out -? Select VM: lhr: 5683606c41098e fdaa:0:3b99:a7b:7e:3155:9844:2 nameless-feather-6339 -Connecting to fdaa:0:3b99:a7b:7e:3155:9844:2... complete -Filesystem 1K-blocks Used Available Use% Mounted on -devtmpfs 103068 0 103068 0% /dev -/dev/vda 8191416 172748 7582856 3% / -shm 113224 0 113224 0% /dev/shm -tmpfs 113224 0 113224 0% /sys/fs/cgroup -/dev/vdb 1011672 2564 940500 1% /data -``` - -The volume is mounted in the directory specified by the `destination` field in the `[mounts]` section of the `fly.toml` file, or the `attach-volume` option for cloned Machines. - -## Related topics - -- [Fly Volumes overview](/docs/reference/volumes/) -- [Manage volume storage](/docs/apps/volume-manage/) -- [`mounts` section](/docs/reference/configuration/#the-mounts-section) in the `fly.toml` Fly Launch configuration file -- [Scale an app with volumes](/docs/apps/scale-count/#scale-an-app-with-volumes) diff --git a/blueprints/autoscale-machines.html.md b/blueprints/autoscale-machines.html.md new file mode 100644 index 0000000000..628ad55bcf --- /dev/null +++ b/blueprints/autoscale-machines.html.md @@ -0,0 +1,117 @@ +--- +title: Autoscale Machines +layout: docs +nav: guides +redirect_from: /docs/blueprints/autoscale-machines-like-a-boss/ +--- + +
+ Illustration by Annie Ruygt of two machines dancing and one standing still +
+ +## Overview + +You have an app with services that's configured to [automatically start +and stop Machines based on traffic demand](/docs/launch/autostop-autostart/). But the traffic to your app changes +significantly during the day and you don't want to keep a lot of stopped +Machines during the period of low traffic. + +This guide will take you through the process of configuring the +[`fly-autoscaler` app](/docs/launch/autoscale-by-metric/) in conjunction with +[Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/) to +always keep a fixed number of Machines ready to be quickly started +by Fly Proxy. + +## Configure autostop/autostart + +First, if you haven't already done so, configure the app to allow Fly Proxy to automatically start and +stop or suspend Machines based on traffic demand. The autostop/autostart settings apply +per service, so you set them within the `[[services]]` or `[http_service]` +sections of `fly.toml`: + +```toml +... +[[services]] + ... + auto_stop_machines = "stop" + auto_start_machines = true + min_machines_running = 0 +... +``` + +With these settings Fly Proxy will start an additional Machine if all the +running Machines are above their concurrency `soft_limit` and stop running +Machines when the traffic decreases. You can set Machines to `"suspend"` rather than +`"stop"`, for even faster start-up, but with some [limitations on the type of Machine](https://community.fly.io/t/new-feature-in-preview-suspend-resume-for-machines/20672#current-limitations-and-caveats-8). + +In the next section you'll configure +and deploy `fly-autoscaler` to ensure that the app always has a spare stopped +Machine for Fly Proxy to start. + +## Configuring and deploying fly-autoscaler + +`fly-autoscaler` is a metrics-based autoscaler that scales an app’s Machines +based on any metric. You can configure it to ensure that there is always +additional Machine available for Fly Proxy to start if the traffic increases. + +First, create a new Fly.io app that will run the autoscaler. + +``` +$ fly apps create my-autoscaler +``` + +Create a deploy token so that the autoscaler app has permissions to scale your +target app up and down: + +``` +$ fly tokens create deploy -a my-target-app +$ fly secrets set -a my-autoscaler --stage FAS_API_TOKEN="FlyV1 ..." +``` + +Create a read-only token so that the autoscaler app has access to a Prometheus instance: + +``` +$ fly tokens create readonly +$ fly secrets set -a my-autoscaler --stage FAS_PROMETHEUS_TOKEN="FlyV1 ..." +``` + +Configure your autoscaler `fly.toml` like this: + +```toml +app = "my-autoscaler" + +[build] +image = "flyio/fly-autoscaler:0.3.1" + +[env] +FAS_PROMETHEUS_ADDRESS = "https://api.fly.io/prometheus/my-org" +FAS_PROMETHEUS_METRIC_NAME = "running_machines" +FAS_PROMETHEUS_QUERY = "count(fly_instance_up{app='$APP_NAME'})" + +FAS_APP_NAME = "my-target-app" +FAS_CREATED_MACHINE_COUNT = "min(running_machines + 1, 10)" +FAS_INITIAL_MACHINE_STATE = "stopped" + +[metrics] +port = 9090 +path = "/metrics" +``` + +With this configuration, the autoscaler will create a new stopped Machine as +soon as all available Machines are running (but never more than 10), and will destroy extra stopped +Machines if more than one Machine is stopped. + +Make sure you are using autoscaler version 0.3.1 or newer for +`FAS_INITIAL_MACHINE_STATE` configuration option to work. + +And finally, deploy the autoscaler, using the `--ha` option to deploy only one Machine: + +``` +$ fly deploy --ha=false +``` + +## Related reading + +- [Autoscale based on metrics](/docs/launch/autoscale-by-metric/) +- [Autostop/autostart Machines](/docs/launch/autostop-autostart/) +- [Autostop/autostart private apps](/docs/blueprints/autostart-internal-apps/) diff --git a/blueprints/autostart-internal-apps.html.md b/blueprints/autostart-internal-apps.html.md new file mode 100644 index 0000000000..9f9c9c96e4 --- /dev/null +++ b/blueprints/autostart-internal-apps.html.md @@ -0,0 +1,123 @@ +--- +title: Autostart and autostop private apps +layout: docs +nav: guides +--- + +
+ Illustration by Annie Ruygt of Frankie the balloon having a picnic with an app and eating strawberries +
+ +## Overview + +You have a private, or internal, app that communicates only with other apps on your [private network](/docs/networking/private-networking/). This private app might be a database, authentication server, or any other "backend" app that you don't want exposed to the public Internet. You want the app's Machines to stop when they're not serving requests from your other apps, and start again automatically when needed. + +To use Fly Proxy autostop/autostart you need to configure services in `fly.toml`, like you would for a public app. But instead of using a public Anycast address, you assign a Flycast address to expose those services only on your private network. + +This guide focuses on using autostop/autostart to control Machines based on incoming requests. But when you use Flycast for private apps you also get other [Fly Proxy features](/docs/reference/fly-proxy/) like geographically aware load balancing. + +Learn more about [Flycast](/docs/networking/flycast/). + +## Create a new private app with a Flycast address + +When you run `fly launch` to create a new app, it automatically assigns your app a public IPv6 address and a shared public IPv4 address. If you know your app won't need to be reachable from the Internet, you can inform Fly Launch to assign a Flycast private IPv6 address instead: + +``` +fly launch --flycast +``` + +Next steps: [Configure and deploy a private app](#configure-and-deploy-a-private-app). + +## Use Flycast for an existing app + +If you already have an app and you want to make it private and use Flycast, it's important to make sure you remove the app's public IP addresses. + +### Add a Flycast address + +``` +fly ips allocate-v6 --private +``` + +### Remove public IP addresses + +List your IPs to check whether your app has public IP addresses: + +``` +fly ips list +``` + +Example output: + +``` +VERSION IP TYPE REGION CREATED AT +v6 2a09:8280:1::2d:1111 public (dedicated) global Sep 1 2023 19:47 +v6 fdaa:2:45b:0:1::11 private global Mar 16 2024 18:20 +v4 66.241.124.11 public (shared) Jan 1 0001 00:00 +``` + +This example app has public IPv4 and IPv6 addresses. These public addresses are automatically assigned to an app with services on the first deploy. + +Copy the public IP addresses and run the `fly ips release` command to remove them from your app: + +``` +fly ips release ... +``` + +For example: + +``` +fly ips release 2a09:8280:1::2d:1111 66.241.124.11 +``` + +Next steps: [Configure and deploy a private app](#configure-and-deploy-a-private-app) below. + +## Configure and deploy a private app + +Whether you're creating a new app or making an existing app private, there are a few things you'll need to check or configure. + +### Add services in your `fly.toml` config file + +If your app was private, you might not have configured an `[http_services]` or `[services]` section in `fly.toml` because you didn't want it reachable through the public Internet. Now that you removed the public IPs, you can safely add services to allow access to the app on your private network and enable Fly Proxy to control Machines and load balance traffic. + +Here's an example `fly.toml` snippet: + +```toml +[http_service] + # the port on which your app receives requests over the 6PN + internal_port = 8081 + # must be false - Flycast is http-only + force_https = false + # Fly Proxy stops Machines based on traffic + auto_stop_machines = "stop" + # Fly Proxy starts Machines based on traffic + auto_start_machines = true + # Number of Machines to keep running in primary region + min_machines_running = 0 + [http_service.concurrency] + type = "requests" + # Fly Proxy uses this limit to determine Machine excess capacity + soft_limit = 250 +``` + +
+**Important:** Set `force_https = false`; Flycast only works over HTTP. HTTPS isn't necessary because all your private network traffic goes through encrypted WireGuard tunnels. +
+ +Learn more about [app configuration](/docs/reference/configuration/) and [Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/). + +### Make sure your app binds to `0.0.0.0:` + +To be reachable by Fly Proxy, an app needs to listen on `0.0.0.0` and bind to the `internal_port` defined in the `fly.toml` configuration file. + +### Deploy the app + +Run `fly deploy` for the configuration changes to take effect. + +Other apps in your organization can now reach your private app using the [Flycast](/docs/networking/flycast/) IP address or the `.flycast` domain. + +## Related reading + +We've talked about Flycast in some past blog posts: + +- [Deploy Your Own (Not) Midjourney Bot on Fly GPUs](https://fly.io/blog/not-midjourney-bot/) +- [Scaling Large Language Models to zero with Ollama](https://fly.io/blog/scaling-llm-ollama/) diff --git a/blueprints/bridge-deployments-wireguard.html.md b/blueprints/bridge-deployments-wireguard.html.md new file mode 100644 index 0000000000..e2dace868a --- /dev/null +++ b/blueprints/bridge-deployments-wireguard.html.md @@ -0,0 +1,119 @@ +--- +title: Bridge your other deployments to Fly.io +layout: docs +sitemap: true +nav: guides +author: xe +categories: + - networking +date: 2024-05-27 +--- + +
+ Illustration by Annie Ruygt of a figure with wings and a ship some balloons and planes +
+ +## Overview + +Sometimes you can recreate production on Fly.io without any issues. Other times you need to be able to incrementally move things over. Fly.io [has private networking](/docs/networking/private-networking/) by default for apps in the same organization, but you can also easily connect your existing external servers to this private network. This lets you use your private network as a way to incrementally move services over. + +Say you have an existing service on AWS or a database with RDS that needs to be accessed over RDS. You can use that AWS instance to pivot traffic from your Fly Machines to those other services. You can also go the other way and access your Fly apps (such as [an instance of Ollama](https://fly.io/blog/scaling-llm-ollama/)) from AWS, your laptop, or any other self-hosted server you have in your arsenal. Or maybe you're making a tool for a support team, and it needs to hit that one database on prem. + +Fly.io private networks use [WireGuard](https://www.wireguard.com/) internally, and when you connect other computers to that network, you do connect to them over WireGuard. WireGuard is used by many people for many reasons, but the primary use is a "site-to-site" VPN like this. This won't route all of your traffic through Fly.io's gateway servers, but it will give you internal access to your private network from anywhere. + +## Weaving the networks together + +Make sure you have [WireGuard](https://www.wireguard.com/install/) installed on your target server. Their [install page](https://www.wireguard.com/install/) has more details, but on an Ubuntu server all you need to do is: + +```docker +sudo apt -y install wireguard +``` + +Once that's done, you're off to the races! + +On your laptop, make the config for the target server with `fly wireguard create`. You'll need the following information: + +- The organization you want to join that server to (such as `personal`, or your company's name). +- The region you want to create the peer in: you should choose [the closest region to the target computer](/docs/reference/regions/). + - If you're really not sure which region is the closest, here's a magic command you can try: `curl -Iso /dev/null -w '%header{fly-request-id}' https://fly.io | cut -d- -f2`. +- The name you want to use for that computer on your Fly network, such as the hostname. +- A path to put the generated WireGuard config, such as `~/fly0.conf` to drop it as `fly0.conf` in your home directory. + +For example, I'm going to create a peer in my personal organization for my server named `phantoon`. I live in Ottawa, so the closest datacenter to me is `yyz`. The command to create my peer will look like this: + +```docker +fly wireguard create personal yyz phantoon ~/fly0.conf +``` + +The `fly0.conf` file contains all the configuration WireGuard needs to set up a tunnel, namely: + +- The private key for the node +- The local IPv6 address the node should use +- The DNS server for your private network +- Addresses, IP ranges, and public keys for the WireGuard gateway + +Copy `fly0.conf` to `/etc/wireguard` on the target computer: + +```docker +scp ~/fly0.conf root@phantoon.local:/etc/wireguard/fly0.conf +``` + +Then connect to the computer (likely over SSH) and enable WireGuard to start on boot: + +```docker +ssh root@phantoon.local +systemctl enable --now wg-quick@fly0.service +``` + +This will create an interface named `fly0`: + +```docker +$ ip addr show fly0 +4: fly0: mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000 + link/none + inet6 fdaa:0:641b:a7b:9285:0:a:1b02/120 scope global + valid_lft forever preferred_lft forever +``` + +Now try to ping the Machines API server `_api.internal`: + +```docker +$ ping _api.internal -c 4 +PING _api.internal (fdaa:0:641b::3) 56 data bytes +64 bytes from fdaa:0:641b::3: icmp_seq=1 ttl=64 time=10.4 ms +64 bytes from fdaa:0:641b::3: icmp_seq=2 ttl=64 time=39.2 ms +64 bytes from fdaa:0:641b::3: icmp_seq=3 ttl=64 time=83.7 ms +64 bytes from fdaa:0:641b::3: icmp_seq=4 ttl=64 time=123 ms +``` + +This means that you can successfully query any [`.internal` addresses](/docs/networking/private-networking/#fly-io-internal-dns) for apps and Machines in your organization. Congrats, you're in! + +## Accessing workloads outside of your private network + +To access a workload outside of your private network (such as a secret store), use the DNS name `peername._peer.internal`. For example, to access the node `phantoon` from your Fly apps, connect to `phantoon._peer.internal`: + +```docker +$ ping phantoon._peer.internal +PING phantoon._peer.internal (fdaa:0:641b:a7b:9285:0:a:1b02) 56 data bytes +64 bytes from phantoon._peer.internal (fdaa:0:641b:a7b:9285:0:a:1b02): icmp_seq=1 ttl=62 time=24.4 ms +64 bytes from phantoon._peer.internal (fdaa:0:641b:a7b:9285:0:a:1b02): icmp_seq=2 ttl=62 time=37.6 ms +64 bytes from phantoon._peer.internal (fdaa:0:641b:a7b:9285:0:a:1b02): icmp_seq=3 ttl=62 time=85.4 ms +64 bytes from phantoon._peer.internal (fdaa:0:641b:a7b:9285:0:a:1b02): icmp_seq=4 ttl=62 time=197 ms +``` + +## Homework + +Getting ping working should be good enough to get you started, but here's some optional homework in case you want to see what you can really do with this: + +- Connect your laptop/workstation to your private network with the same flow (you may need to install the GUI WireGuard app if you use Windows or macOS). +- Expose Prometheus metrics on port 9195 of your app and then grab the current metrics with `curl yourapp.internal:9195/metrics` on your laptop. +- Install Postgres on your laptop somehow. Configure your app to connect to that Postgres database on your laptop over WireGuard. +- Connect a few other computers to your private network. Install a Minecraft server on one of them and play together. + +Hint: you may need to allow private network addresses through your firewall. Check the documentation of your firewall tool of choice and allow traffic from `fly0` through as if it's an "internal" interface. + +## Related Reading + +- [Jack into your private network with WireGuard](https://fly.io/docs/blueprints/connect-private-network-wireguard/) A guide showing how to connect a laptop or external server to your Fly.io private network via WireGuard. +- [Private Networking](https://fly.io/docs/networking/private-networking/) Overview of Fly’s internal IPv6 mesh (6PN), how apps talk to each other privately, and how WireGuard‑based tunnels integrate. +- [Custom Private Networks](https://fly.io/docs/networking/custom-private-networks/) How to isolate tenants, apps, or services by creating separate private networks inside your organization — relevant when bridging external infrastructure. \ No newline at end of file diff --git a/blueprints/cell-based.html.md b/blueprints/cell-based.html.md new file mode 100644 index 0000000000..7700190cf6 --- /dev/null +++ b/blueprints/cell-based.html.md @@ -0,0 +1,255 @@ +--- +title: Cell-based architecture +layout: docs +nav: guides +author: rubys +categories: + - shared-nothing + - cell-based + - machines + - routing +toc: false +status: alpha +published: false +--- + +Previously know as +[shared-nothing-architecture](https://en.wikipedia.org/wiki/Shared-nothing_architecture), +the concept was rediscovered and rebranded as +[cell-based](https://docs.aws.amazon.com/wellarchitected/latest/reducing-scope-of-impact-with-cell-based-architecture/what-is-a-cell-based-architecture.html) +by Amazon. + +This blueprint shows you how to implement a scenario where each user of your +application is assigned a new Machine. These Machines, by default, will +automatically stop when not in use and be restarted when accessed, so the +primary expense will be for volumes and actual usage. + +
+**Warning**: if you follow this example, applications will be deployed without +their databases being replicated. This exposes your application to volume and +Machine failures. Backups become your responsibility. Some approaches to +addressing this requirement are listed at the bottom of this +page. +
+ +The example given here will be based on Rails, but the approach applies to all +frameworks. + +## Start With a Single Machine Application + +If you don't have one, the following will do: + +``` +rails new blog --css tailwind +cd blog +bin/rails generate scaffold Post title:string body:text +fly launch --name blog-$USER-$RANDOM +``` + +Feel free to substitute a different name for your application. + +## Configure Your Routes + +Modify `config/routes.rb`. Wrap your routes with: + +```ruby +scope ENV.fetch("http://github.com/FLY_MACHINE_ID", "") do +... +end +``` + +If present, move `get "up"` line outside of this block. + +Uncomment the `root` line. + +Your final result will look something like this: + +```ruby +Rails.application.routes.draw do + scope ENV.fetch("http://github.com/FLY_MACHINE_ID", "") do + resources :posts + # Define your application routes per the DSL in https://guides.rubyonrails.org/routing.html + + # Defines the root path route ("/") + root "posts#index" + end + + # Reveal health status on /up that returns 200 if the app boots with no exceptions, otherwise 500. + # Can be used by load balancers and uptime monitors to verify that the app is live. + get "up" => "rails/health#show", as: :rails_health_check +end +``` + +What this will do is place your entire application, minus the health check +endpoint, into a namespace when deployed to fly.io. Access to your application +will need to be prefixed by your Machine id. When run in environments where +the `FLY_MACHINE_ID` environment variable is not set, no prefix is required. + +You can get a list of Machine ids using: + +``` +fly machines list -q +``` + +## Starting a Second Machine + +In order to prepare for a multiple Machine deployment, we are going to need to +ensure that each request is routed to the correct Machine. We can accomplish +this via middleware that makes use of the +[fly-replay](https://fly.io/docs/networking/dynamic-request-routing/#the-fly-replay-response-header) +response header. + +Place the following into `config/initializers/fly_router.rb`: + +```ruby +if ENV["FLY_MACHINE_ID"] + class FlyInstanceRouter + def initialize(app) + @app = app + end + + def call(env) + segments = env["PATH_INFO"].split('/') + instance = segments[1] + + if instance =~ /^[0-9a-f]{14}$/ and instance != ENV["FLY_MACHINE_ID"] + return [409, {"Fly-Replay" => "instance=#{instance}"}, [""]] + end + + @app.call(env) + end + end + + Rails.application.config.middleware.use(FlyInstanceRouter) +end +``` + +Deploy this change using `fly deploy`, then use the following command to create +a second Machine: + +``` +fly machine clone +``` + +At this point, you have two instances of the blog application, each on its own +Machine, with its own volume and sqlite3 database. + +Note that [fly machine clone](https://fly.io/docs/flyctl/machine-clone/) has a +number of options that can be used to specify such things as the region, vm +sizes, and whether the volume requires a unique zone. + +Regions, in particular, can enable you to put both applications _and_ data +close to users. + +## Optimizing your routes + +This section is optional, and depends on a unique feature of Rails' +[Hotwire](https://hotwired.dev/) implementation. + +At this point, many requests will end up making two hops: first to a nearby +Machine then to the desired Machine. This increases latency of responses, +particularly when it is necessary to wake two Machines to process the first +request. + +Not much can be done to avoid this for the first request, but subsequent +requests can route to the correct Machine using the `fly-force-instance-id` +request header. + +Place the following into `app/javascript/controllers/fly_router_controller.js`: + +```js +import { Controller } from "@hotwired/stimulus" + +// Connects to data-controller="fly-router" +export default class extends Controller { + connect() { + console.log('fly router connected') + this.instance = this.element.dataset.instance; + + document.documentElement.addEventListener( + 'turbo:before-fetch-request', + this.beforeFetchRequest + ) + } + + disconnect() { + document.documentElement.removeEventListener( + 'turbo:before-fetch-request', + this.beforeFetchRequest + ) + } + + beforeFetchRequest = event => { + console.log('injecting ' + this.instance); + event.detail.fetchOptions.headers['fly-force-instance-id'] = this.instance; + } +} +``` + +Finally, modify the `` line in `app/views/layouts/application.html.erb`: + +```erb + data-instance="<%= ENV['FLY_MACHINE_ID'] %>" data-controller="fly-router"<% end %>>> +``` + +Deploy this change using `fly deploy`. + +## Next steps + +The above merely amounts to a proof of concept. A production application will +likely make use of one or more of the following: + + * While putting the Machine id into the URL path is effective, + it isn't very ergonomic. A better solution would be have a registry of + paths and their associated Machines. + * Implementing routing in middleware gets the job done, but + separating this out to a separate process using an application like + [`nginx`](https://nginx.org/en/) can have a number of advantages: + * `fly-replay` is limited to request payloads of one megabyte or less, + which would affect features like file uploads. A [reverse + proxy](https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/) + might be a better solution. + * support for custom subdomains as an alternative to path namespaces and + scopes. + * authentication and serving static assets can be performed outside of your + application. + * Add-ons like [Phusion Passenger](https://www.phusionpassenger.com/) can + enable you to run multiple instances of your application in one Machine. + * Monitoring becomes more crucial when you have hundreds or even dozens of + independent Machines. Fly.io will combine your logs and while this + addresses many problems, it doesn't help deal with logs that are missing due + to application or network problems. Consider having applications emit a + heartbeat, and write monitoring software that looks for missing heartbeats + and reports them as issues. [Sentry](https://fly.io/docs/monitoring/sentry/) + can help here. + * While applying updates when all of the services for a single Machine are + self contained is an easier problem then upgrading potentially + interdependent services running live in production, it does come at a cost: + a blue/green deployment is not possible so a focus on reducing startup times + is necessary. + * Consider building an administrative web based interface for your + application. User registration and Machine assignments are likely coupled + in your workflow. While Fly.io provides a Machines API, + you can go a long way with old-fashioned scripting using the flyctl command. + +See [Showcase +Architecture](https://github.com/rubys/showcase/blob/main/ARCHITECTURE.md) for +an example of an application running in production that implements many of +these ideas. + +## Backups + +As mentioned above, backups are crucial. Items to explore: + + * [LiteFS](https://fly.io/docs/litefs/) - Distributed SQLite. By itself, it + supports failover situations naturally. Additional [disaster + recovery](https://fly.io/docs/litefs/backup/) options are available. + * Sqlite3 databases are just files. Build a separate application to host + backups and have your application periodiacally POST copies there. + * [Rsync](https://rsync.samba.org/) is a utility available with Linux + distributions that can be used to efficiently copy changes between Machines. + * Run multiple Machines per cell so that you get the full benefits of + traditional high availability configurations. If you have implemented an + admin web UI, you can automate the deployment of new clusters easily. diff --git a/blueprints/connect-private-network-wireguard/index.html.md b/blueprints/connect-private-network-wireguard/index.html.md new file mode 100644 index 0000000000..fa33cdd38d --- /dev/null +++ b/blueprints/connect-private-network-wireguard/index.html.md @@ -0,0 +1,147 @@ +--- +title: Jack into your private network with WireGuard +layout: docs +sitemap: true +nav: guides +author: xe +categories: + - networking +date: 2024-06-14 +--- + +

+ +Every [Fly.io](http://Fly.io) organization comes with a [private network](https://fly.io/docs/networking/private-networking/) that lets all your apps connect to each other. This is super convenient when you need to have microservices call each other’s endpoints or use [Flycast](/docs/networking/flycast/) to let your private apps turn off when you’re not using them. However, this isn’t just limited to your apps. You can jack into this network with WireGuard. + +This blueprint shows you how to create a WireGuard peer to your private network and connect to it so that you can access it from anywhere. + +## Prerequisites + +To get started, you need to have the following: + +- A [fly.io](http://fly.io) account +- `flyctl` installed (https://fly.io/docs/hands-on/install-flyctl/) +- The [WireGuard client](https://www.wireguard.com/install/) installed + +## Steps + +To create a WireGuard peer, you need the following information: + +- The organization you want to create the peer in, such as your personal organization. +- The [Fly.io region](/docs/reference/regions/) that’s closest to you. +- The name of the peer, such as your computer’s hostname. +- A filename to save the configuration to. + +``` +$ fly wg create --help +Add a WireGuard peer connection to an organization + +Usage: + fly wireguard create [org] [region] [name] [file] [flags] +``` + +You can figure out your list of organizations with `fly orgs list`: + +``` +$ fly orgs list +Name Slug Type +---- ---- ---- +Xe Iaso personal PERSONAL +devrel-demos devrel-demos SHARED +``` + +You can figure out which region is nearest you with `fly platform regions` or by checking the [regions page](https://fly.io/docs/reference/regions/) in the documentation: + +``` +$ fly platform regions +``` + +With all this in mind, let’s assemble the command. Each of these steps is going to build up the command, but don't hit enter until it's all done. + +Start with the base command `fly wireguard create`: + +``` +$ fly wireguard create +``` + +I want to create this in my personal organization, so I’ll enter in personal for the organization name. + +``` +$ fly wireguard create personal +``` + +I’m in Ottawa, so I’m using the Montreal region `yul`. + +``` +$ fly wireguard create personal yul +``` + +My computer’s hostname is Camellia, so I’ll use that as the peer name. + +``` +$ fly wireguard create personal yul camellia +``` + +Finally I want to save this as camellia.conf so that WireGuard can load it. + +``` +$ fly wireguard create personal yul camellia camellia.conf +``` + +Then run the command (you can hit enter now), and once it’s done open up the WireGuard app. + +Import the tunnel from the configuration file and then turn it on. macOS may prompt if you want the WireGuard app to manage VPN connections. If it does, hit accept, otherwise you won’t be able to get into your network. + + + +The above video shows how you import a WireGuard config on macOS. Here's a summary of the steps: + +- Click on the "Import tunnel(s) from file" button. +- Select the configuration file you saved earlier. +- Click on the "Activate" button. +- macOS will ask you if you want to allow the WireGuard app to manage VPN connections. Click "Allow" if you trust it. + +On a Linux system you can use [wg-quick](https://www.man7.org/linux/man-pages/man8/wg-quick.8.html+external): + +```sh +wg-quick up camellia.conf +``` + +To test the connection, ping `_api.internal` (on macOS you need to run `ping6 _api.internal` because it's an IPv6 address): + +``` +$ ping6 _api.internal -c4 +PING6(56=40+8+8 bytes) fdaa:3:9018:a7b:9285:0:a:602 --> fdaa:3:9018::3 +16 bytes from fdaa:3:9018::3, icmp_seq=0 hlim=64 time=9.741 ms +16 bytes from fdaa:3:9018::3, icmp_seq=1 hlim=64 time=49.103 ms +16 bytes from fdaa:3:9018::3, icmp_seq=2 hlim=64 time=97.667 ms +16 bytes from fdaa:3:9018::3, icmp_seq=3 hlim=64 time=14.726 ms + +--- _api.internal ping6 statistics --- +4 packets transmitted, 4 packets received, 0.0% packet loss +round-trip min/avg/max/std-dev = 9.741/42.809/97.667/35.111 ms +``` + +To test the connection with an instance of [Ollama](https://ollama.com), you can fire it up with the following commands in an empty folder (don't forget to delete it after!): + +``` +fly launch --from https://github.com/fly-apps/ollama-demo --no-deploy +fly ips allocate-v6 --private +fly deploy +``` + +Then you can set the `OLLAMA_HOSTNAME` environment variable to the hostname of the instance you just created (it will be something like `xe-ollama.flycast`), and then run the following commands: + +``` +export OLLAMA_HOST=http://xe-ollama.flycast +ollama run llama3 "Why is the sky blue? Explain in a single sentence." +``` + +And then the model will reply with something like this: + +> The sky appears blue because of a phenomenon called Rayleigh scattering, where shorter wavelengths of light (like blue and violet) are scattered more than longer wavelengths (like red and orange) by tiny molecules of gases like nitrogen and oxygen in the Earth's atmosphere. + +And there you go! You're in your private network and can access all your apps and Machines like they were right next to you. diff --git a/blueprints/connect-private-network-wireguard/wireguard-activate.mp4 b/blueprints/connect-private-network-wireguard/wireguard-activate.mp4 new file mode 100644 index 0000000000..96cbdaeaa7 Binary files /dev/null and b/blueprints/connect-private-network-wireguard/wireguard-activate.mp4 differ diff --git a/blueprints/connecting-to-user-machines-http-flow.svg b/blueprints/connecting-to-user-machines-http-flow.svg new file mode 100644 index 0000000000..148bd9de69 --- /dev/null +++ b/blueprints/connecting-to-user-machines-http-flow.svg @@ -0,0 +1 @@ +

Arrives at

Sets fly-replay

replay

match port 443

Arrives at

Sets fly-replay

replay

match port 9090

Public HTTP Request

Router App: llm-router.fly.dev:443

Fly Proxy

User App: llm-user-1234

public:443

Private HTTP Request

Management Gateway: llm-gateway.flycast:9090

management:9090

 \ No newline at end of file diff --git a/blueprints/connecting-to-user-machines.html.md b/blueprints/connecting-to-user-machines.html.md new file mode 100644 index 0000000000..290a74cdb6 --- /dev/null +++ b/blueprints/connecting-to-user-machines.html.md @@ -0,0 +1,164 @@ +--- +title: Connecting to User Machines +layout: docs +nav: guides +date: 2025-04-02 +--- + +
+ Illustration by Annie Ruygt of a bird holding hands with a computer +
+ +## Overview + +When running machines for end users, a common challenge is efficiently managing and routing requests to these machines. This guide outlines the recommended approach for connecting to user machines on Fly.io. + +## Typical Stack + +- **Coordinator**: Your app that manages machines on behalf of end users. This can also function as the Router. +- **Router**: An app that relays requests to user machines. This can be the same as the Coordinator. +- **User App**: Apps assigned to specific users on isolated networks. +- **User Machine**: Machine(s) in those apps. + +## HTTP Services + +A typical setup includes two services: +- A **management** HTTP service (`port 9090`) for tasks such as: + - Sending LLM generated code changes + - Health checks + - Metrics requests +- A **public** HTTP service (`port 4443`) for: + - Live Previews + - LLM <-> MCP Server Connectivity + - Deployed User Apps + +### HTTP Request Processing + +![HTTP Request Flow](/docs/blueprints/connecting-to-user-machines-http-flow.svg) + +This diagram illustrates how HTTP requests are processed: +- Public HTTP requests arrive at your router app, which issues a fly-replay to a specific user app. The Fly Proxy then replays the request to the target app, matching the service based on the inbound request port. +- Private HTTP requests arrive at an internal management gateway, which is not accessible from the internet. The same replay process gets that request to the user app on the specified port, matching the service based on the inbound request port. + +## Using fly-replay + +The recommended mechanism for connecting to user machines over HTTP is the `fly-replay` response header. This works by instructing our global proxy to reroute the request to a specific machine. If the machine is not running, the proxy will start it. This approach works for any kind of HTTP request, including WebSockets. Since our proxy manages the entire request, your router app is no longer a point of failure after it sends the `fly-replay` header. + +### Port Matching + +When using fly-replay, the proxy matches the incoming request's port to the target service's public port. For example: +- A request to port 80 will be routed to a service configured with `port: 80` +- A request to port 9090 will be routed to a service configured with `port: 9090` + +This means your router app needs to receive requests on the same ports that your target services are configured to use. For example, to connect to a management API on port 9090, the router must receive the request on port 9090. + +### Private Network Router Example + +The [private-network-router](https://github.com/fly-apps/private-network-router) example shows how to build a router that can handle multiple services. It uses flycast to keep management APIs private while still allowing routing to them. The router: + +1. Receives requests on any port +2. Determines the target app and machine based on the request +3. Sends a `fly-replay` response header with parameters like `app` and `instance` to route the request to the correct app and machine. For more details on the parameters, see the [fly-replay documentation](https://fly.io/docs/networking/dynamic-request-routing/#the-fly-replay-response-header). + +This pattern allows you to: +- Keep management APIs private by running them with flycast +- Route requests to multiple services from a single router +- Maintain security by not exposing management ports directly to the internet + +## Authentication + +Validation of replayed requests should happen within the user machine. We recommend using the `state` mechanism in fly-replay for this purpose: + +1. **Create a preshared key for each user app** and set it as an app secret. +2. When issuing a replay, set the header: `fly-replay: app=,state=`. +3. On the user machine, parse the `fly-replay-src` header. This header contains fields such as `instance`, `region`, `t`, and `state` (see [fly-replay documentation](https://fly.io/docs/networking/dynamic-request-routing/#the-fly-replay-response-header)). +4. Check the `state` value in `fly-replay-src` against the preshared key for the app. If it matches, the request is validated. + +See the [TypeScript example below](#example-authenticating-a-replayed-request-in-typescript) for a practical implementation of this validation. + +## Antipatterns + +- **flycast, .fly.dev, and Dedicated IPs**: These are meant for permanent apps that need to communicate with each other. They and involve more moving parts than simply routing with `fly-replay`. More moving parts mean more potential for issues. +- **TCP or UDP**: Avoid using raw TCP or UDP when communicating with end user machines. If necessary, consider running Tailscale containers alongside your code. + +## Example Service Configuration + +Here's an example service configuration for a machine that includes both a management service and a standard public HTTP service: + +```json +{ + "services": [ + { + "name": "management", + "internal_port": 9090, + "ports": [ + { + "port": 9090, + "handlers": ["http"] + } + ] + }, + { + "name": "public", + "internal_port": 8080, + "ports": [ + { + "port": 80, + "handlers": ["http"] + } + ] + } + ] +} +``` + +This configuration sets up: +- A management service listening internally on port 9090 and exposed on port 9090 +- A public service listening internally on port 8080 and exposed on port 80 +Both services will automatically start when needed and stop when idle. + +### Example: Authenticating a Replayed Request in TypeScript + +Here's a simple example of how you might authenticate a replayed request in a TypeScript HTTP handler using the `fly-replay-src` header and a preshared key: + +```typescript +// Example preshared key for the app (in practice, load from env or secret store) +const PRESHARED_KEY = process.env.PRESHARED_KEY; + +function parseFlyReplaySrc(header: string | undefined) { + if (!header) return {}; + return Object.fromEntries( + header.split(',').map(pair => { + const [key, value] = pair.split('='); + return [key.trim(), value?.trim()]; + }) + ); +} + +async function authenticateReplay(request: Request): Promise { + const flyReplaySrc = request.headers.get('fly-replay-src'); + const params = parseFlyReplaySrc(flyReplaySrc); + return params.state === PRESHARED_KEY; +} + +// Example usage in a fetch handler +addEventListener('fetch', (event) => { + event.respondWith(handleRequest(event.request)); +}); + +async function handleRequest(request: Request): Promise { + if (!(await authenticateReplay(request))) { + return new Response('Unauthorized', { status: 401 }); + } + // ...handle the authenticated request... + return new Response('OK'); +} +``` + +This code checks the `fly-replay-src` header, parses out the `state` value, and compares it to the preshared key. If it matches, the request is authenticated. + +## Related reading + +- [Per‑User Dev Environments with Fly Machines](/docs/blueprints/per-user-dev-environments/) Find out how to run isolated, on‑demand environments for each user. +- [Observability for User Apps](/docs/blueprints/observability-for-user-apps/) Learn about how to gather logs and telemetry from per‑user apps/machines once you’ve routed traffic to them. +- [Dynamic Request Routing with fly‑replay](/docs/networking/dynamic-request-routing/) Learn about the `fly‑replay` header and dynamic request routing. \ No newline at end of file diff --git a/blueprints/connecting-to-user-machines.mmd b/blueprints/connecting-to-user-machines.mmd new file mode 100644 index 0000000000..140b97d23c --- /dev/null +++ b/blueprints/connecting-to-user-machines.mmd @@ -0,0 +1,10 @@ +graph TD; + A[Public HTTP Request] -->|Arrives at| B[Router App: llm-router.fly.dev:443] + B -->|Sets fly-replay| C[Fly Proxy] + C -->|replay| D[User App: llm-user-1234] + D -->|match port 443| E[public:443] + + F[Private HTTP Request] -->|Arrives at| G[Management Gateway: llm-gateway.flycast:9090] + G -->|Sets fly-replay| C + C -->|replay| D + D -->|match port 9090| H[management:9090] \ No newline at end of file diff --git a/blueprints/deno-kv-litefs.html.markerb b/blueprints/deno-kv-litefs.html.markerb new file mode 100644 index 0000000000..59ae1c52d2 --- /dev/null +++ b/blueprints/deno-kv-litefs.html.markerb @@ -0,0 +1,152 @@ +--- +title: Deno KV with LiteFS Cloud +layout: docs +published: false +nav: firecracker +author: jesse +date: 2024-05-28 +--- + +<%= partial "/docs/partials/docs/litefs_sunset" %> + +## Problem + +If you're porting an app from Deno Deploy to Fly.io, you may hit a bit of a snag: if it was built with [Deno KV](https://deno.com/blog/kv), a _traditionally_ FoundationDB backed K/V store made for Deno Deploy, there's no obvious way to easily run Deno KV at scale on Fly.io. You _could_ use [denoland/denokv](https://github.com/denoland/denokv) on Fly.io, but due to an alignment of the stars there happens to be an easier way. + +Deno KV can actually be backed by an SQLite DB stored on disk in a cache folder, and you can specify the path of this DB with the parameter to `Deno.openKv( )`. + +This means you can do something like this to get Deno KV to use an arbitrary SQLite DB: + +```typescript +const kv = await Deno.openKv("/any/path/i/want.db"); +``` + +Now, Fly.io has some special features for SQLite users, namely [LiteFS Cloud](https://fly.io/docs/litefs/speedrun), a distributed file system that transparently replicates and backs up SQLite databases across all your instances. The magic here is that you can just treat it like a local on-disk SQLite database but behind the curtain it’s doing all the work to replicate your DB. + +## Setting up LiteFS + +Firstly, you need to add a `litefs.yml` file to your project, and make sure it gets included in the Dockerfile. + +Here’s an example: + +```yml +fuse: + dir: "/litefs" + +data: + dir: "/var/lib/litefs" + +exit-on-error: false + +proxy: + addr: ":8080" + target: "localhost:8081" + db: "db" + passthrough: + - "*.ico" + - "*.png" + +exec: + - cmd: "deno run -A main.ts" + +lease: + type: "consul" + advertise-url: "http://${hostname}.vm.${fly_app_name}.internal:20202/" + candidate: ${FLY_REGION == PRIMARY_REGION} + promote: true + + consul: + url: "${FLY_CONSUL_URL}" + key: "litefs/${FLY_APP_NAME}"\ +``` + +This is 99% the [sample config](https://github.com/superfly/litefs-example/blob/main/fly-io-config/etc/litefs.yml), only changing the exec command to a Deno one. + +
+It's not immediately clear based on the example config, but your app needs to listen on "target" (:8081) and your service/http_service in fly.toml needs to listen on "addr" (:8080). This is because part of LiteFS acts as a proxy, so you need the fly-proxy to send requests to LiteFS which then forwards them on to your app. + +If you’re not able to change your application’s port, make sure `target` is set up correctly then change `addr` and `fly.toml` to something other than `:8080`. + +
+ +Once you've added the config, you just need to make your app look for the SQLite DB in the right location. For Deno KV, that looks like this: + +```typescript +const kv = await Deno.openKv(Deno.env.get("DB_LOCATION")); +``` + +Where `DB_LOCATION` is set to `/litefs/my.db` in `fly.toml` under `[env]`. + +Now, if you’re deploying to Fly.io, you’re almost ready to go. [Here’s where to look if you’re not running on Fly.io](https://fly.io/docs/litefs/getting-started-docker/). + +## Creating a LiteFS Cloud cluster + +For a more in-depth article, check out [Getting Started with LiteFS on Fly.io](https://fly.io/docs/litefs/getting-started-fly/). + +Head over to the [LiteFS section](https://fly.io/dashboard/personal/litefs) of the dashboard and create a cluster, **make a note of the auth token** (you’ll need it later). + +### Dockerfile + +You need to add the dependencies for LiteFS to your Dockerfile: + +```docker +# for alpine-based images +RUN apk add ca-certificates fuse3 sqlite + +# or for debian/ubuntu-based images +RUN apt-get update -y && apt-get install -y ca-certificates fuse3 sqlite3 +``` + +And copy in the LiteFS binary: + +```docker +COPY --from=flyio/litefs:0.5 /usr/local/bin/litefs /usr/local/bin/litefs +``` + +And update the `ENTRYPOINT`/`CMD`: + +```docker +ENTRYPOINT litefs mount +``` + +Now you're ready to launch your Fly app! + +```bash +# Create a volume +$ fly volumes create litefs --size 10 + +# Create your app without deploying +$ fly launch --no-deploy + +# Attach the Fly.io provided Consul server to your app. +$ fly consul attach + +# Set the secret for LiteFS cloud, replace the example value with your token from earlier +$ fly secrets set LITEFS_CLOUD_TOKEN=yoursecrettoken +``` + +Update your `fly.toml` to mount the LiteFS volume: + +``` +[mounts] + source = "litefs" + destination = "/var/lib/litefs" +``` + +Aaannnndddd, Deploy! + +``` +fly deploy +``` + +Your reward for running all those commands should be a running app with LiteFS! Let us know if you run into friction using LiteFS with Deno KV or any other client, you can reach out on [the Fly Community](https://community.fly.io/). + +### Scaling + +Deploying just one node using LiteFS doesn’t actually _do much_, so try scaling outwards! You can add a few Machines in regions close to your users like this: + +``` +fly scale count 3 -r yyz,ewr,waw +``` + +This will automatically add volumes and Machines to [scale your app](/docs/apps/scale-count/#scale-an-apps-regions) out and around the world! diff --git a/blueprints/going-to-production-with-healthcare-apps.html.md b/blueprints/going-to-production-with-healthcare-apps.html.md new file mode 100644 index 0000000000..aa78b1096d --- /dev/null +++ b/blueprints/going-to-production-with-healthcare-apps.html.md @@ -0,0 +1,187 @@ +--- +title: Going to Production with Healthcare Apps +layout: docs +nav: guides +redirect_from: /docs/blueprints/going-to-production-with-hipaa-apps +--- + +
+ Illustration by Annie Ruygt of a balloon doctor using stethescope on an app +
+ +## Overview + +Fly.io was built by security researchers from the ground up to be both productive and secure, making it a great home for HIPAA-compliant production healthcare applications for productive teams that ship often. + +This guiide runs a developer or operations engineer through the process of evaluating Fly.io's security for HIPAA healthcare apps, launching a pilot application, signing a Business Associate Agreement (BAA), and deploying to production. + +## HIPAA and Fly.io Primer + +The Health Insurance Portability and Accountability Act (HIPAA) sets the standard for protecting sensitive patient data. Any company that processes protected health information (PHI) must ensure that all the required physical, network, and process security measures are in place and followed. + +Fly.io takes a "principle of least privilege" approach to security. Here are the highlights of what healthcare app developers get out of the box on Fly.io: + +### Data Protection and Encryption + +HIPAA requires that PHI be encrypted both in transit and at rest to prevent unauthorized access: + +- **In Transit**: Fly.io uses [WireGuard](https://fly.io/blog/our-user-mode-wireguard-year/) to encrypt data as it moves between networks, ensuring compliance with HIPAA's transmission security requirements. +- **At Rest**: Fly.io ensures data at rest is secured on [encrypted NVMe user volumes](/docs/volumes/), aligning with the encryption standards required by HIPAA for storage security. + +### Access Control + +Access to PHI must be limited to authorized personnel only, which is managed through: + +- **Network Isolation**: Each application on Fly.io runs within its own private network by default, reducing the risk of unauthorized access. +- **Secrets Management**: Fly.io provides a [secrets manager](/docs/apps/secrets/) for storing and handling API credentials and environment variables securely, ensuring that sensitive information is accessible only to authorized applications. +- **Isolated Application Environments**: Fly.io allows organizations to create separate organizations for different environments, ensuring that production environments are isolated from test and staging environments. +- **Workload Isolation:** [Docker workloads on Fly.io are isolated at the kernel hardware level through the Firecracker VM runtime](https://fly.io/blog/fly-machines/). These workloads are hosted in secure data centers on Fly.io's dedicated hardware, ensuring HIPAA compliance by minimizing the risk of unauthorized access and maintaining strict control over PHI data. + +### Audit Controls + +HIPAA demands that you implement hardware, software, and procedural mechanisms that record and examine activity in systems containing PHI: + +- **Logging and Monitoring**: Fly.io allows organizations to monitor and log access and usage, helping comply with HIPAA’s information system activity review requirements. +- **Version-Controlled Configuration**: Fly.io uses a Dockerfile and fly.toml configuration file for managing application deployments. These files can be checked into version control systems, like Github, allowing for comprehensive audit trails of infrastructure changes. This capability supports HIPAA's mandates for traceability and accountability in the modification and management of systems handling PHI. + +### Data Integrity + +To comply with HIPAA, entities must ensure that PHI is not altered or destroyed in an unauthorized manner: + +- **Data Storage and Backup**: Fly.io’s infrastructure provides robust data integrity controls, including [regular snapshots](/docs/volumes/volume-manage/) of [encrypted volumes](/docs/volumes/overview/#volume-encryption) to prevent data loss. +- **Reproducible Deployments**: Fly.io's use of Docker ensures that application deployments are reproducible and can be locked down to a mostly read-only state. This reduces the likelihood of unauthorized modifications to application environments, meeting HIPAA's requirements for protecting PHI from unauthorized alteration or destruction. + +### Physical Security + +Physical access to data centers where PHI is stored must be controlled: + +- **Data Centers**: Fly.io hosts services in secure data centers with restricted access, surveillance, and environmental controls, ensuring the physical security of the hardware that stores and processes PHI. +- **Regional Data Control**: Fly.io allows users to control the [physical region](https://fly.io/docs/reference/regions/) where data is stored and processed using simple commands, facilitating compliance with data residency requirements of HIPAA by ensuring PHI is stored within designated secure regions, such as the United States. + +There's more detail at [https://fly.io/docs/about/healthcare](https://fly.io/docs/about/healthcare/) and [https://fly.io/security](https://fly.io/security). If you have additional questions about Fly.io's security infrastructure & practices, please [reach out](mailto:sales@fly.io). + +## Launch pilot application + +The fastest way to deploy a pilot application is to run through the [Fly.io Speedrun](https://fly.io/speedrun/). + +If your free demo account doesn't have enough resources for your pilot application, you may [upgrade to a paid plan](https://fly.io/docs/about/pricing/) or [contact us](mailto:sales@fly.io) to make other arrangements. + +### Specify deployment regions + +Fly automatically detects which of its 30+ regions around the globe is closest to you as your default data region. + +If the default region is not where you want to host your application, run `fly platform regions` to see the most recent list of regions. + +```bash +fly platform regions +NAME CODE GATEWAY GPUS CAPACITY LAUNCH PLAN+ + +North America +Ashburn, Virginia (US) iad ✓ ✓ 164 +Chicago, Illinois (US) ord ✓ 385 +Dallas, Texas (US) dfw ✓ 426 +Los Angeles, California (US) lax ✓ 635 +San Jose, California (US) sjc ✓ ✓ 2399 +Secaucus, NJ (US) ewr 233 +Toronto, Canada yyz ✓ 70 +# ... Lots more regions... +``` + +Then select your desired region: + +``` +$ fly regions add ord +``` + +And remove undesired regions as well: + +```bash +$ fly regions remove sin +``` + +Your application will automatically be deployed to the regions specified and communications between regions happen over a private, encrypted WireGuard network. + +### Add API keys and credentials to the Secrets Manager + +Fly.io's [secret manager stores](/docs/apps/secrets/) your application's API credentials and secrets in a secure vault. The contents remain encrypted and are only available from running Machine instances as environment variables. + +### Talk to our solution architects if you have questions + +If at any time during your evaluation of Fly you have questions about the infrastructure, security, or service terms you can ask in the community forums or [email Fly.io with your questions](mailto:sales@fly.io). + +## Deploy to production + +Once you've evaluated Fly.io and have a BAA, it's time to go to production. The remainder of this guide will walk you through how to setup an isolated production environment, control access, and deploy your application. + +### Sign a Business Associate Agreement (BAA) + +When you're ready to start deploying HIPAA apps, you'll need to do some paperwork (don't worry, we use digital signatures) to make sure everything is compliant. + +- [Choose the Compliance Package](https://fly.io/compliance) that includes HIPAA/BAA documents. +- Sign in to the dashboard and request a signed BAA at [https://fly.io/dashboard/personal/compliance](https://fly.io/dashboard/personal/compliance) or [contact us](mailto:sales@fly.io) and we'll help. + +### Provision an isolated production environment + +Ensure the production environment to be separate from the test and staging environments used to test this application. + +Fly uses "organizations" to control access to "applications". Since production environments require their own access control, we'll create a new organization that will contain all of our production applications. + +```bash +# Replace $MYORG with the name of your organization +$ fly orgs create $MYORG-production +``` + +You'll walk through the steps of creating a new organization within Fly.io. You'll also have to add a new payment method and select the appropriate plan for the new organization. If you need help during this process or have questions about how a plan spans multiple organizations, [please reach out](mailto:sales@fly.io). + +### Invite team members to production environment + +Grant access to team members who may have access to the production environment: + +```bash +$ fly orgs invite somebody@$MYORG.com --org $MYORG-production +``` + +### Launch application + +Next, ensure you have a `fly.toml` file for each environment: + +```bash +$ cp fly.toml fly.staging.toml +$ mv fly.toml fly.production.toml +``` + +Run `fly launch` to provision the production environment. + +```bash +# Replace $MYAPPNAME with the name of your app +$ fly launch --name $MYAPPNAME-production --path fly.production.yml --org $MYORG-production +An existing fly.toml file was found for app $MYAPPNAME +? Would you like to copy its configuration to the new app? (y/N) y +``` + +You'll be asked to copy the configuration file. Select `y` assuming you want to use the same settings from the `fly.toml` file that was formerly staging. You'll want to commit these `fly.toml` files to your repo. + +If you run into problems during or after deploy you can run `fly logs --path fly.production.yml` to see errors your application may be logging. + +### Create SSL certificate and custom domain + +Next, point a domain name to the new production environment via `fly certs`. + +```bash +$ fly certs add myappname.com --path fly.production.yml +``` + +The output of the command includes the information you need to update DNS records to point to the application on Fly.io + +### Deploy application + +Finally run `fly deploy --path fly.production.yml` to deploy your application, then open your application in your browser to ensure that it's up and running. + +## Wrap-up + +Fly.io is a secure, productive platform for deploying HIPAA-compliant healthcare applications. The platform provides the necessary security controls and features to ensure that PHI is protected in accordance with HIPAA requirements while giving development teams the flexibility to ship often and scale quickly. + +## Related reading + +- [Healthcare apps on Fly.io](/docs/about/healthcare/) Overview of how Fly.io supports HIPAA‑compliant apps, including encryption, network isolation, and audit‑controls. +- [Going to production checklist](/docs/apps/going-to-production/) A general production‑readiness checklist for apps on Fly.io. \ No newline at end of file diff --git a/blueprints/index.html.md b/blueprints/index.html.md new file mode 100644 index 0000000000..02af045ab7 --- /dev/null +++ b/blueprints/index.html.md @@ -0,0 +1,68 @@ +--- +title: Guides (Blueprints) +layout: docs +toc: true +nav: guides +--- + +A growing library of guides for running, designing, and deploying all kinds of apps on Fly.io. Whether you're just getting started or looking for new ideas, these are real-world patterns you can learn from and build on. + +## Architecture Patterns + +Guides for the structure your app on Fly.io. Layouts, tradeoffs, moving parts. + +- [Resilient apps use multiple Machines](/docs/blueprints/resilient-apps-multiple-machines/) +- [Getting Started with N-Tier Architecture](/docs/blueprints/n-tier-architecture/) +- [Shared Nothing Architecture](/docs/blueprints/shared-nothing/) +- [Session Affinity (a.k.a. Sticky Sessions)](/docs/blueprints/sticky-sessions/) +- [Multi-region databases and fly-replay](/docs/blueprints/multi-region-fly-replay/) +- [Deploying Remote MCP Servers](/docs/blueprints/remote-mcp-servers/) + + +## Deployment & Developer Workflow + +Stuff you set up once and adjust when you ship code. Includes previews, base images, staging, and Docker wrangling. + +- [Seamless Deployments on Fly.io](/docs/blueprints/seamless-deployments/) +- [Rollback Guide](/docs/blueprints/rollback-guide/) +- [Git Branch Preview Environments on Github](/docs/blueprints/review-apps-guide/) +- [Staging and production isolation](/docs/blueprints/staging-prod-isolation/) +- [Per-User Dev Environments with Fly Machines](/docs/blueprints/per-user-dev-environments/) +- [Using base images for faster deployments](/docs/blueprints/using-base-images-for-faster-deployments/) +- [Managing Docker Images with Fly.io's Private Registry](/docs/blueprints/using-the-fly-docker-registry/) + + +## Networking & Connectivity + +Connecting things. Public apps, private services, bridging deployments, SSH, and WireGuard. + +- [Run private apps with Flycast](/docs/blueprints/private-applications-flycast/) +- [Jack into your private network with WireGuard](/docs/blueprints/connect-private-network-wireguard/) +- [Bridge your other deployments to Fly.io](/docs/blueprints/bridge-deployments-wireguard/) +- [Connecting to User Machines](/docs/blueprints/connecting-to-user-machines/) +- [Run an SSH server](/docs/blueprints/opensshd/) + +## Scaling, Performance & Observability + +Make it fast. Make it reliable. Monitor what's happening. + +- [Observability for User Apps](/docs/blueprints/observability-for-user-apps/) +- [Autoscale Machines](/docs/blueprints/autoscale-machines/) +- [Autostart and autostop private apps](/docs/blueprints/autostart-internal-apps/) +- [Setting Hard and Soft Concurrency Limits](/docs/blueprints/setting-concurrency-limits/) +- [Using Fly Volume forks for faster startup times](/docs/blueprints/volume-forking/) +- [Going to Production with Healthcare Apps](/docs/blueprints/going-to-production-with-healthcare-apps/) + + +## Background Jobs & Automation + +How to run periodic tasks, long-running jobs, infrastructure automation, and the things that run when you’re asleep. + +- [Building Infrastructure Automation without Terraform](/docs/blueprints/infra-automation-without-terraform/) +- [Deferring long-running tasks to a distributed work queue](/docs/blueprints/work-queues/) +- [Task scheduling guide with Cron Manager and friends](/docs/blueprints/task-scheduling/) +- [Crontab with Supercronic](/docs/blueprints/supercronic/) + + + + diff --git a/blueprints/infra-automation-without-terraform.html.md b/blueprints/infra-automation-without-terraform.html.md new file mode 100644 index 0000000000..0625b42aac --- /dev/null +++ b/blueprints/infra-automation-without-terraform.html.md @@ -0,0 +1,173 @@ +--- +title: Building Infrastructure Automation without Terraform +layout: docs +nav: guides +author: kcmartin +date: 2025-08-05 +--- + +
+ Illustration by Annie Ruygt of a Frankie the balloon working on automation at his computer +
+ +## Overview + +We don’t have a Terraform provider anymore, and that’s intentional. It’s not a great fit for how our platform works. This guide is about how to get a Terraform-like experience using the tools that do fit. + +If you're used to defining infrastructure declaratively, using templated variables, repeatable workflows, a single source of truth, you can absolutely have that on Fly.io. But you’ll get there by thinking a little differently: using tools built around our actual primitives instead of trying to bend them to fit a plan/apply model. If you're looking for a good path forward, you've got options. + +This guide walks through two options for automating Fly.io deployments without Terraform. + +The first uses `flyctl` and GitHub Actions. It’s what we recommend for most users: push to `main`, your app gets deployed. No infra state to manage, no VMs to track. Just code and Git. + +The second uses the Machines API directly. It’s more work, but gives you full control over each virtual machine. You’ll need this if you’re doing custom orchestration like rolling deploys across regions, temporary environments, or using fine-tuned scaling logic. + +We’ll show you how both approaches work and help you figure out which one makes sense for your setup. + +--- + +## The easy way: `flyctl` + GitHub Actions + +This is the path most Fly.io apps take, and it’s what we recommend. You write code, you push to `main`, your app gets deployed. It’s quick to set up, and it abstracts away the VM lifecycle stuff unless and until you want it. + +Here’s the gist: + +1. Get your app running on Fly.io +1. Push the code to a GitHub repo +1. Wire up a GitHub Actions workflow that calls `flyctl deploy` + +That’s it. Here’s what setting that up looks like. + +### Step 1: Generate a deploy token + +``` +fly tokens create deploy --app +``` + +Treat that token like a password. Copy it somewhere safe—you’ll need it for GitHub. + +### Step 2: Add it to your GitHub repo + +In GitHub: + +- Go to your repo > Settings > Secrets and variables > Actions +- Add a new secret named `FLY_API_TOKEN` +- Paste in the deploy token + +### Step 3: Write the workflow file + +Drop this into `.github/workflows/fly.yml` in your repo: + +``` +name: Fly Deploy + +on: + push: + branches: + - main # or your default branch + +jobs: + deploy: + name: Deploy app + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: superfly/flyctl-actions/setup-flyctl@master + - run: flyctl deploy --remote-only + env: + FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} +``` + +Push that file, and your next `git push` to `main` will kick off a deployment. You can watch it happen in the Actions tab. + +The `--remote-only` flag tells `flyctl` to build your image on Fly.io’s remote builders, so your CI environment doesn’t need Docker. + +From here on out, you’ve got continuous deployment. Push to `main`, app goes live. + +A realistic example: you’re running a Rails app backed by Postgres and Redis, deployed to `iad` and `lhr`. It lives in a monorepo, and you want each push to `main` to trigger a deployment. The `flyctl` + GitHub Actions setup gets you continuous deployment. You'll handle the supporting infrastructure imperatively (think databases, IP addresses, scaling): `fly pg create`, `fly redis create`, `fly scale count`, `fly ips allocate`, and so on. The platform keeps track of what you’ve configured, so you don’t need to re-declare it every time. + +
+**Tip:** If you want a repeatable setup, you can script your commands into a bash file. Just be aware: that script will be linear and imperative, not declarative or idempotent like a Terraform plan. +
+ +--- + +## The harder way: The Machines API + +This is the route for teams who were doing more than `terraform apply`. If you had logic layered into your infra automation like canary deploys, ephemeral environments, controlling exactly which VM runs where, you’ll want to look at the Machines API. It’s lower-level, but more powerful. Think of it as Fly.io’s bare-metal interface. No `plan`, no `state`, no `graph`, just HTTP. + +You can use it to: + +- Spin up new Machines in specific regions +- Roll out changes a Machine at a time +- Build custom scaling logic +- Tear down and rebuild environments on demand + +This is what Fly.io’s own orchestration tools use under the hood. You can spin up Machines, wire them together, and orchestrate them across regions, just like we do. You've got all the same knobs and levers we use internally, and you can build exactly the kind of workflow your setup needs. + +### What you're working with + +The Machines API is a REST interface. It speaks JSON over HTTPS. You authenticate using a [Fly.io API token](https://fly.io/docs/security/tokens/), ideally a deploy token scoped to your app or organization. Pass it as a `Bearer` token in the `Authorization` header. If you've used GitHub Actions to deploy with `flyctl`, it's the same kind of token. + +Each Fly App is a namespace. You can create as many Machines (VMs) as you want in an App. Each Machine runs a Docker image, has a region, some resource settings, and a lifecycle: create → start → stop → destroy. + +You don’t need special tooling to use it. `curl` works. So does `fetch`, `requests`, `httpie`, or whatever HTTP client you like in Go or Rust or Python. Everything you might script with `flyctl`, like creating databases, setting scale counts, can also be done via the Machines API. It’s more verbose, but gives you full control and makes it easier to integrate infrastructure setup into a larger orchestration flow. + +### Example: spin up a Machine with curl + +``` +export FLY_APP_NAME="your-app-name" +export FLY_API_TOKEN="your-fly-token" + +curl -i -X POST \ + -H "Authorization: Bearer ${FLY_API_TOKEN}" \ + -H "Content-Type: application/json" \ + "https://api.machines.dev/v1/apps/$%7BFLY_APP_NAME%7D/machines" \ + -d '{ + "region": "sjc", + "config": { + "image": "nginx", + "guest": { + "cpu_kind": "shared", + "cpus": 1, + "memory_mb": 256 + } + } + }' +``` + +The response includes the ID of the new Machine. You can then start, stop, or destroy it by hitting `/machines/{id}` with the appropriate verb. + +### When to reach for the API + +Most apps don’t need this. But if you’re building something dynamic, like spinning up short-lived browser-based dev environments or launching ephemeral multiplayer game servers, then you’ll want this kind of control. + +Some of our favorite real-world uses: + +- A “run this one function on a new machine” pattern, demoed in [this JS repo](https://github.com/fly-apps/fly-run-this-function-on-another-machine). It boots a new Machine in-region, runs a job, and tears it down. Works like serverless, but with a full VM and no cold starts. +- A minimal Go client for the Machines API, [`sosedoff/fly-machines`](https://github.com/sosedoff/fly-machines), has helpers to create, update, and destroy Machines programmatically. If you’re replacing Terraform logic with your own Go-based orchestration, this is a good lightweight example. For something more comprehensive, check out our official fly Go SDK, [`superfly/fly-go`](https://github.com/superfly/fly-go), for tighter integration with the platform. + +The Machines API doesn’t lock you into a flow, we just give you the primitives. + +**Still not sure? Here's the comparison.** + +| Feature | `flyctl` + GitHub Actions | Machines API | +| --- | --- | --- | +| Best for | Standard apps, CI/CD | Custom infra, dynamic environments | +| Setup time | Fast | Slower | +| Complexity | Low | High | +| Control | Abstracted (`deploy`, `scale`) | Full (`create`, `start`, `destroy`) | +| Official support | Fully supported & recommended | Fully supported & documented | + +## One last thing + +We know it sucks when a tool you relied on gets deprecated. Terraform made a lot of things feel clean and declarative. But when it comes to the Machines API specifically, Terraform was always working at a bit of a mismatch since it tries to express an inherently imperative lifecycle in a declarative model. The Machines API gives you fine-grained control and flexibility that didn’t map cleanly to Terraform's abstractions. With a little effort, you can build exactly what you need, and avoid being boxed in by someone else’s idea of what an app should look like. + +If you're rebuilding that tooling now, let us know what you're working on. We might be able to help, or at least learn something from it. + +## Related Reading + +- [Continuous Deployment with flyctl and GitHub Actions](https://fly.io/docs/launch/continuous-deployment-with-github-actions/) Walk‑through on wiring CI/CD with Fly.io + GitHub Actions. +- [Access Tokens for Fly.io](https://fly.io/docs/security/tokens/) Deep dive into token scopes, permissions, and best practices for automation workflows. +- [Working with the Machines API](https://fly.io/docs/machines/api/working-with-machines-api/) Reference guide for automating at the VM/“machine” level (the lower‑level alternative to flyctl). +- [App Configuration (`fly.toml`)](https://fly.io/docs/reference/configuration/) Details on how your app config file works, which is often part of your automation setup. \ No newline at end of file diff --git a/blueprints/multi-region-fly-replay.html.md b/blueprints/multi-region-fly-replay.html.md new file mode 100644 index 0000000000..5f933657bd --- /dev/null +++ b/blueprints/multi-region-fly-replay.html.md @@ -0,0 +1,114 @@ +--- +title: Multi-region databases and fly-replay +layout: docs +nav: guides +--- + +
+ Illustration by Annie Ruygt of Frankie looking out over the solar system +
+ +## Overview + +You want to run your app and database in multiple [regions](/docs/reference/regions/) close to your users, and deploying database read replicas will give you better performance on read requests. This is a good solution for apps with read-heavy database workloads. But you also want writes to be efficient, despite needing to write to a primary database on the other side of the world. + +This guide will help you understand the concepts to implement an app, with a primary database and read replicas in multiple regions, that uses the `fly-replay` response header to re-route write requests to the primary region. Consider using this guide when: + +- Your app's users are in more than one geographic region and you want low latency for reads in all regions. +- Your app has a request distribution of less than 50% writes. +- You don't want the architecture complications and potential for conflicts that come with running a multi-primary database cluster. + +## Fly.io was made for multi-region apps + +Fly.io already routes requests to the closest region where your app is deployed. Using regional database read replicas can further reduce latency since the Fly Machine (our fast-launching VM) receiving the request can connect to a database instance in the same region. Read replicas also reduce the overall load on the primary database. This multi-reader single-writer setup is great for read-heavy apps. + +When your app receives write requests, you can use the `fly-replay` response header to forward the whole HTTP request to the primary region, which is often faster than connecting directly to the primary database from other regions. This is especially true when a request to an endpoint requires multiple database queries to complete; there would be extra latency on each query from a secondary region compared to a local connection in the primary region. + +## How it works + +The [`fly-replay` response header](/docs/networking/dynamic-request-routing/) instructs Fly proxy to redeliver (replay) the original request to another region or Machine in your app, or even another app in your organization. In this case, you’ll be replaying write requests to the Machine in the primary region. Using `fly-replay` to replay write requests is a general pattern that can be applied in most languages and frameworks for databases with one primary and multiple read replicas. + +In the following diagram, the app is running one Machine in each of three regions. The primary region is Chicago, and this is where the read/write primary database resides. There are Machines in two other regions, Rio de Janeiro and Amsterdam, each of which has a read replica. This example uses three regions for simplicity, but you could deploy in more than three regions and have more than one Machine per region connecting to the same read replica. + +
+ Three Machines with attached databases in 3 regions: Chicago is the primary region with a writable database, while Rio de Janeiro and Amsterdam have read only replica databases. Arrows pointing from the secondary Machines to the primary Machine show the direction of HTTP requests redelivered using the fly-replay response header. Arrows pointing from the primary database to the read replicas indicate syncing of data. +
+ +
+**Note:** To illustrate the `fly-replay` concept in our diagram, we show the replayed HTTP request going directly from Machines in Rio de Janeiro and Amsterdam to the Machine in Chicago. In real life, Fly Proxy routes the request back through an edge node first. The cost of this routing is small, but if extreme efficiency is important for your use case, you can run your app in more regions to mitigate that. +
+ +Your app is running on Fly.io. The database can also run on Fly.io—in which case the app and database regions will match—or on another provider where you can pick the region closest to the Fly.io region of your Machines. + +## How to make it work for your app + +Your app needs logic to: + +1. **Read from the corresponding regional replica (or primary!) when a Machine receives a read request.** +1. **Always replay write requests to the primary region rather than connecting to the primary database from another region.** + +### Read from the corresponding regional replica + +Your app has a configurable primary region and you'll also host your primary database there or, in the case of a separate database provider, as close as possible to it. The primary region is exposed in the `PRIMARY_REGION` environment variable on every Machine in your app. + +Each Machine exposes the region in which it's running in the `FLY_REGION` environment variable. + +Your app can check the `FLY_REGION` against the `PRIMARY_REGION`, and modify the `DATABASE_URL`, `DB_HOST`, or other database variables when the region codes don't match. In some cases this just means changing a port ([as for Postgres](/docs/postgres/advanced-guides/high-availability-and-global-replication/#connecting-to-read-replicas)) and in other cases it will be more complex if the database requires different variables for each read replica (like in this [Laravel example](/laravel-bytes/multi-region-laravel-with-planetscale/)). + +### Replay write requests to the primary region + +Your app can detect write requests and send a response with the `fly-replay` header that tells Fly Proxy to replay the whole request to the Fly Machine in the primary region. + +#### Detect write requests + +First, you'll need a way to detect write requests so that you can forward them to the primary database. How you detect write requests will depend on your app and database. With Postgres databases, probably the simplest way to detect write requests is by [catching “read only transaction” errors](/docs/postgres/advanced-guides/high-availability-and-global-replication/#detecting-write-requests) when you attempt to write to a read replica. + +You can also set up routes in middleware if your language provides support for splitting read and write connections. Or your app can handle GET, POST, DELETE, PUT, and PATCH requests differently, again likely in your app's middleware. For example, assuming HTTP requests using GET are read requests and everything else is a write request will work well most of the time. + +#### Send a response with the fly-replay header + +When your app detects a write request, it should send a response that contains only the `fly-replay` header with the [region code](/docs/reference/regions/) of your primary region like so: `fly-replay: region=`. Fly Proxy will replay the whole request to the specified region. The code would look something like this: + +```yaml +# This code should be in the handler for non-GET requests to the endpoint you +# want to send to the primary region for write to the primary database. +if os.environ("PRIMARY_REGION") != os.environ("FLY_REGION"): + # This machine is not in the primary region + response.headers = {"fly-replay": f"region={os.environ("PRIMARY_REGION")}"} + return response +else: + # This machine *is* in the primary region + # Handle request normally, write to database. +``` + +## Challenges and limitations + +### Write-heavy apps shouldn't use read replicas + +Don't use regional read replicas if your app's requests are more than 50% write requests. In that case, you'd want to look into a multi-region primary or similar solution. + +### Reading your own writes + +There will be short periods of time when data is not consistent between replicas and the primary. If reading your own writes immediately is a requirement for your app, then you'll need to consider how to handle replication. + +You can set up health checks to monitor replication status and fail when replication is not complete. + +Another useful pattern is to poll the database until the data is replicated. The Fly Postgres [LSN module](https://github.com/superfly/fly_postgres_elixir/tree/main/lib/lsn+external) has an example of how to poll with a [Postgres stored procedure](https://github.com/superfly/fly_postgres_elixir/blob/main/lib/migrations/v01.ex+external). You can reuse this pattern from any database library. + +A less efficient method would be for your app to catch the errors thrown when records don't exist and then retry failed requests. + +## Implementation resources + +Example implementations of this multi-region database with `fly-replay` pattern can give you a head start: + +- [Fly Ruby gem](https://github.com/superfly/fly-ruby+external) for running database replicas alongside your app instances in multiple regions. +- [Fly Postgres Elixir library](https://github.com/superfly/fly_postgres_elixir+external) for geographically distributed Elixir applications using Ecto and PostgreSQL in a primary/replica configuration on Fly.io. + +## Related reading + +We've covered multi-region databases with `fly-replay` in some past blog posts and in our docs: + +- [Globally Distributed Postgres](https://fly.io/blog/globally-distributed-postgres/) +- [Multi-Region Laravel with PlanetScale](https://fly.io/laravel-bytes/multi-region-laravel-with-planetscale/) +- [Run Ordinary Rails Apps Globally](https://fly.io/ruby-dispatch/run-ordinary-rails-apps-globally/) +- [Dynamic Request Routing](/docs/networking/dynamic-request-routing/) \ No newline at end of file diff --git a/blueprints/n-tier-architecture.html.md b/blueprints/n-tier-architecture.html.md new file mode 100644 index 0000000000..17f7dd9600 --- /dev/null +++ b/blueprints/n-tier-architecture.html.md @@ -0,0 +1,130 @@ +--- +title: Getting Started with N-Tier Architecture +layout: docs +nav: guides +author: kcmartin +date: 2025-08-29 +--- + +
+ Illustration by Annie Ruygt of a bird slicing a loaf of bread-think app layers +
+ +## Introduction: What is n-tier architecture? + +When people talk about “**n-tier architecture**,” they’re describing a way of splitting an app into layers (or “tiers”) that each have a specific job. The “**n**” just means there could be two, three, or more tiers depending on how you slice things. + +--- + +## Defining tiers + +- **Presentation tier**: the UI/frontend of your app — HTML/CSS/JS in a browser, or a mobile app interface — that the user interacts with +- **Web tier**: the app servers/backend that [handle requests](/docs/networking/dynamic-request-routing/) and run business logic +- **Data tier**: stores, manages and provides access to application data + +--- + +## N-Tier Examples + +### Two tiers + +**web app + database**. This is the most common case on Fly.io: your app servers run on Fly Machines and connect to a shared Postgres cluster. + +### Three tiers + +**frontend + backend + database**. You’ll see this if you have a separate frontend (like a React SPA served from a CDN, or a mobile app) that talks to your Fly backend. + +### Mapping to Fly.io + +- **Web tier** → your app running on Fly Machines +- **Data tier** → [Fly Managed Postgres](/docs/mpg/) (our hosted Postgres that handles the messy parts for you) +- **Optional presentation tier** → a separate frontend app (web or mobile) that talks to your Fly backend + +The idea is simple but powerful: the web tier doesn’t keep state. All shared state lives in the data tier, so you can add or remove servers at will without breaking anything. It’s the architecture you want if you care about **scalability** (handling more traffic by adding machines) and **reliability** (requests keep flowing even if one machine goes down). + +--- + +## Why use n-tier on Fly.io? + +- **Scale with one command**: Add more Web/App machines whenever traffic spikes. +- **Keep requests flowing**: Fly’s proxy automatically balances load across your Machines. +- **Let the database do its job**: With [Managed Postgres](/docs/mpg/), you don’t need to worry about setup, backups, or failover. + +You could use Redis, Tigris, or other data stores here too, but if you’re not sure, start with Managed Postgres. It’s the default choice for good reason. + +--- + +## Quickstart: Deploying a n-Tier App + +Let’s spin up a simple n-tier setup. + +### 1. Launch your app + +First, deploy your app to Fly: + +```bash +fly launch +fly deploy +``` + +This gives you the **web tier**. + +--- + +### 2. Add a database (Managed Postgres) + +Now create your **data tier** with [Fly Managed Postgres](/docs/mpg/): + +```cmd +fly mpg create +``` + +This will: + +- Provision a new Postgres cluster in the same region as your app to keep latency to a minimum (though you can choose another region). +- Configure monitoring, daily backups, and automatic failover. + +To connect your app, you’ll need to add the database credentials as secrets. You can do this from the Fly.io Dashboard, or with `flyctl`: + +```cmd +fly secrets set DATABASE_URL=postgres://:@/ +``` + +Most frameworks (Rails, Django, Node, etc.) will automatically read `DATABASE_URL` from the environment. Once set, your app is ready to talk to Postgres. + +More details on creating and connecting a managed Postgres database can be found [here](/docs/mpg/create-and-connect/). + +--- + +### 3. Scale out your web tier + +If traffic grows, add more Machines: + +```cmd +fly scale count 3 +``` + +[Fly’s proxy](/docs/reference/fly-proxy/) takes care of spreading requests across the machines (load balancing). + +**Note:** Scaling works best when your web tier is **stateless**. Any shared data like file uploads, user sessions/cookies, or cached state should live in your data tier (usually Managed Postgres, sometimes object storage or Redis). If your framework defaults to local disk or memory for these, configure it to use a shared store instead. + +--- + +## Finishing up + +At this point you’ve got: + +- **Optional presentation tier**: a separate frontend app, if you use one (delivered to browsers or devices) +- **Web tier**: Fly Machines running your app (stateless and load balanced) +- **Data tier**: [Fly Managed Postgres](/docs/mpg/) + +Every request can hit any Machine, and all Machines share the same Postgres. That’s an n-tier architecture in action, and it’s the foundation for most reliable web apps. + +--- + +## Related reading + +- Explore the features of [Fly Managed Postgres](/docs/mpg/) +- Find out how to [scale apps on Fly](/docs/launch/scale-count/) +- Read about [Fly networking](/docs/networking/) +- Learn more about [n-tier architecture](https://en.wikipedia.org/wiki/Multitier_architecture#Three-tier_architecture) diff --git a/blueprints/observability-for-user-apps.html.md b/blueprints/observability-for-user-apps.html.md new file mode 100644 index 0000000000..692c2dcee8 --- /dev/null +++ b/blueprints/observability-for-user-apps.html.md @@ -0,0 +1,121 @@ +--- +title: Observability for User Apps +layout: docs +nav: guides +date: 2025-05-26 +--- + +
+ Illustration by Annie Ruygt of a printed photo depicting a bird and a ghost backpacking through a forest +
+ +## Overview + +When you run a platform for users to execute their own code on Fly.io—[think per-user development environments](/docs/blueprints/per-user-dev-environments/) or AI/LLM apps—it's a good idea to have real-time observability for those user apps. You might _also_ want to stream logs from those apps back to the original developer (your end-user), so they can see what their code is doing. + +Fly.io offers built-in telemetry based on [**NATS**](https://nats.io/) that you can tap into to achieve this. At a high level, Fly.io captures everything your app writes to stdout and funnels it into an internal NATS log stream. A proxy in front of this NATS cluster ensures that each organization can only access its own applications' logs. Both the `flyctl logs` command and the our web dashboard's "Live Logs" use this pipeline under the hood. We can use the same system! By connecting a NATS client within your Fly organization's [private network](/docs/networking/private-networking/), we can subscribe to log subjects and stream logs out to wherever we need, in real-time. + +## Deploying the Fly Telemetry Forwarder + +If you want a bird's‑eye view for your own operations team, Fly provides an open source reference app called [**fly-telemetry**](https://github.com/superfly/fly-telemetry) that you can deploy to quickly get an observability stack running in your Fly organization. This app acts as a telemetry forwarder: it subscribes to the platform's NATS logs (and metrics) streams and stores the data for you. It includes a lightweight time-series database (VictoriaMetrics for metrics, VictoriaLogs for logs) and a Grafana instance with pre-configured dashboards, so you can explore your apps' logs and metrics visually. + +In an empty directory, run: + +```console +fly launch --from https://github.com/superfly/fly-telemetry \ + --yes \ + --copy-config \ + --org "$ORG" \ + --env ORG="$ORG" \ + --secret ACCESS_TOKEN="$(fly tokens create readonly "$ORG")" \ + --flycast +``` + +After the deployment finishes, browse to the app's private URL [through WireGuard](/docs/blueprints/connect-private-network-wireguard/) or `fly proxy` and you have instant Grafana dashboards. + +This is handy for SREs, but not necessary for your end‑users. + +--- + +## Streaming Fly app logs to your end users + +Streaming live logs straight to your users gives them insight into what their code is doing—developers see exactly what the runtime sees, in real time, without leaving your product or juggling extra tooling. + +### Connecting to the NATS Log Stream + +Every Fly organization has access to **NATS** which carries the logs for its apps. The server is available at the well-known address `[fdaa::3]:4223`. To connect to this endpoint, you must do so from within the Fly network—for example, from code running in a Fly app (like your telemetry forwarder or router) or from your local machine over a WireGuard connection. + +Using the NATS JavaScript client, you could connect as follows: + +```javascript +import { connect, StringCodec } from "nats"; + +const nc = await connect({ + servers: "[fdaa::3]:4223", + user: "sandwich-expert", // your Fly org slug (example) + pass: process.env.ACCESS_TOKEN // your Fly access token (read-only) +}); + +// Now nc is a NATS connection that can subscribe to log subjects. +``` + +### Authentication + +When connecting from a Fly app in your org, you'll need to supply an access token. To generate one manually, run `fly tokens create readonly ` and store it as an app secrets. The NATS proxy will accept the token as the password and your org name as the username. + +### Log Subjects and Filtering + +Logs in the NATS stream are organized by **subject**. The subject format for Fly logs is: + +```console +logs... +``` + +For example, if you have an application named `panini-786` and it's running in region `dfw`, you might see subjects like `logs.panini-786.dfw.d896d7c609dd68` (where `d896d7c609dd68` is the Machine ID). You can subscribe at varying levels of granularity using NATS wildcards: + +| Subject Pattern | Description | +|----------------|-------------| +| `logs.>` | Subscribe to **all logs** from all apps in your org (every region, every Machine) | +| `logs..*.>` | Subscribe to all logs from a specific app (across all regions and Machine) | +| `logs.*..>` | Subscribe to all logs from a specific region | +| `logs...` | Subscribe to a single Machine (full exact subject) | + +In practice, to stream all logs for a given user's app, you'll subscribe to the subject pattern `logs..>` – this will capture any log line from any Machine of that app, in any region. + +### A Real-World Example + +[**NATstream**](https://natstream.fly.dev/) is a reference app that streams logs from your Fly apps right back to the browser. It connects to NATS and turns those events into a live feed at a simple HTTP `/logs` endpoint using Server-Sent Events (SSE). Developers could log in to your product and watch their app's logs flow in real time, without needing separate logging tools or the Fly CLI. The app keeps things high-level and lightweight—there's no deep framework complexity here, just a straightforward example of hooking into Fly's log pipeline and broadcasting events to the browser. + +
+ Screenshot of the NATstream open source app UI +
+ +You can fork it as a foundation for your own developer-facing live log streaming service, or rip out the relevant parts and build this logic into your existing app, using the NATstream implementation as a guide. More robustly, you could run a separate log "router" app that handles authentications and streaming on a per user, per app basis. This would be especially useful if streaming logs to a CLI. + +--- + +## Building a Central Log Router Service + +Create a Fly "log router" app that sits between Fly's NATS telemetry stream, your product and your developers. Because each customer already has their own Fly app, every user's logs appear on a unique subject (`logs..>`), so multi-tenancy is solved by design—subscribe only to the subject that matches the requesting developer's app and you'll never mix data between users. + +
+**Note**: Your router service should enforce that a user can only request the logs for the app(s) they own. This means your routing logic needs an access control check (e.g. based on the authenticated user's ID matching the app name or an internal mapping). Fortunately, if you've structured your platform as one Fly app per user, it's straightforward to maintain a mapping from user account to Fly app name and use that for authorization. +
+ +On startup the router opens one NATS connection with an org-scoped read-only token. When a developer signs in through your dashboard or CLI, the router authenticates that user, determines their Fly app name, and, if not already listening, subscribes to that app's subject. As log entries arrive, the router forwards them over a live channel—typically a websocket or Server-Sent Events stream—directly to the developer's browser or CLI. + +The router itself is a Fly app that does the following: + +- On startup, connect to the NATS log stream using the org-wide credentials (as shown earlier). +- Wait for developers to request or connect for log streaming (e.g., a developer opens a web UI to view their app's live logs, or runs a command to view logs). +- When a request for a specific app's logs is received, subscribe to that app's log subject (if not already subscribed). +- Stream the incoming log messages to the requesting developer in real-time. +- Ensure that each developer only receives their own app's logs. The router should use the mapping of app ← → user to publish the right data to the correct output channel (and not mix different users' data). + +This approach uses one NATS subscription _per user app_. The messages are forwarded verbatim to the user's websocket in this case (as JSON strings), but you could transform them (for example, format them nicely or filter out certain internal logs) before sending. On the client side, you would simply read from the websocket or SSE stream and append new log lines to a console view. + +## Related reading + +- [Logging overview](/docs/monitoring/logging-overview/) How logs flow from your Fly apps into the platform’s NATS pipeline. +- [Logs API options](/docs/monitoring/logs-api-options/) Guide with approaches for tapping directly into structured logs programmatically. +- [fly-telemetry](https://github.com/superfly/fly-telemetry) Lightweight reference implementation for quick, out-of-the-box observability for deployments on Fly.io. \ No newline at end of file diff --git a/blueprints/opensshd.html.md b/blueprints/opensshd.html.md new file mode 100644 index 0000000000..dadc86d502 --- /dev/null +++ b/blueprints/opensshd.html.md @@ -0,0 +1,217 @@ +--- +title: Run an SSH server +layout: docs +nav: guides +author: rubys +categories: + - SSH +date: 2024-01-14 +redirect_from: /docs/app-guides/opensshd/ +--- + +
+ Illustration by Annie Ruygt of a terminal window, a computer and some numbers +
+ +## Overview + +A number of tools allow you to interact with your server over SSH. These tools are useful for tasks such as +copying files ([rsync](https://rsync.samba.org/+external), [scp](https://en.wikipedia.org/wiki/Secure_copy_protocol+external), [sshfs](https://github.com/libfuse/sshfs#sshfs+external)), +editing ([emacs](https://www.gnu.org/software/emacs/manual/html_node/emacs/Remote-Files.html+external), [vim](https://www.vim.org/scripts/script.php?script_id=1075+external), [vscode](https://code.visualstudio.com/docs/remote/ssh+external)), and +deployment ([ansible](https://www.ansible.com/+external), [github actions](https://github.com/marketplace/actions/ssh-deploy+external), and [kamal](https://kamal-deploy.org/+external)). + +One way to use these tools is to [set up a wireguard VPN](https://fly.io/docs/reference/private-networking/#install-your-wireguard-app) and +[issue a new SSH credential](https://fly.io/docs/flyctl/ssh-issue/). This may be impractical for some use cases (example: github actions). + +As an alternative, you can configure and deploy an SSH server on your machine(s). This guide will walk you through the process. + +Before proceeding, a **caution**: unless you are **certain** that all of the clients will access this service through IPv6, you will need a [dedicated IPv4](https://fly.io/docs/reference/services/#dedicated-ipv4) address. + +## Install and configure opensshd + +Most Docker images are ultimately based on Debian, so the following will work. Adjust as necessary for other operating systems (example: [Alpine](https://www.alpinelinux.org/+external)). + +```Dockerfile +RUN apt-get update \ + && apt-get install -y openssh-server \ + && cp /etc/ssh/sshd_config /etc/ssh/sshd_config-original \ + && sed -i 's/^#\s*Port.*/Port 2222/' /etc/ssh/sshd_config \ + && sed -i 's/^#\s*PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config \ + && mkdir -p /root/.ssh \ + && chmod 700 /root/.ssh \ + && mkdir /var/run/sshd \ + && chmod 755 /var/run/sshd \ + && rm -rf /var/lib/apt/lists /var/cache/apt/archives +``` + +Notes: + * This runs `sshd` internally on port 2222. This is because by default `sshd` listens on all network interfaces, + but fly.io machines already are listening on port 22 on the private network (ipv6) interface, so sshd will + either complain or fail to start. Additionally, `fly deploy` will attempt to verify that your application is + listening on the interfaces your application exports, and will incorrectly treat fly's listening on port 22 as + evidence that your application is up and running. + * Password Authentication is disabled. This avoids unnecessary prompts. We will be using SSH keys instead. + * The above assumes `root` user. If your application is running under a different user, replace `/root` with the + home directory of that user (example: `/home/rails`). + +## Map internal port 2222 to external port 22 + +Add the following to `fly.toml`: + +``` +[[services]] + internal_port = 2222 + protocol = "tcp" + auto_stop_machines = true + auto_start_machines = true + [[services.ports]] + port = 22 +``` + +This section is needed even if the internal and external ports are the same. + +Notes: + +* `internal_port` needs to match the port you selected in the previous step. +* `port` can be any available port. `22` is the [default port](https://www.ssh.com/academy/ssh/port+external) for SSH, + and the one that most applications expect to be used. +* Like with your web server port, your server can be configured to spin down when idle and restart when accessed. + Feel free to adjust `auto_stop_machines` and `auto_start_machines` if your needs differ. + +## Start the openssh server + +There are a [number of ways to run multiple processes](../multiple-processes/). The most straightforward +way to start sshd before your application. Locate the `ENTRYPOINT` in your Dockerfile. If you don't have +one, create a script in your application directory, name that script as the ENTYRPOINT, and make it +executable. + +An example of such a script: + +```bash +#!/bin/bash -e + +/usr/sbin/sshd + +exec "$@" +``` + +Note: the above needs to be run as root. If your Dockerfile specifies another `USER`, you can work around this +by installing and configuring `sudo` and then removing sudo access before running your main process. + +For example, if your userid is `rails`, you would add the following to your Dockerfile: + +```Dockerfile +RUN apt-get install -y sudo && \ + echo "%rails ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers +``` + +And then update your entrypoint script: + +```bash +sudo /usr/sbin/sshd +sudo sed -i "/^%rails/d" /etc/sudoers +``` + +The above commands will start `sshd` as root, then remove sudo access from your unprivileged userid. + +## Upload your SSH key + +First, locate your SSH key. You can create a new key using [`ssh-keygen`](https://www.ssh.com/academy/ssh/keygen+external), +Or you can use an existing one: look inside the `.ssh` folder in your home directory for a file with a name like `id_rsa.pub`. + +Once located, there are multiple ways to proceed. Following you will find two ways. Depending on the framework your +application uses, you may have other ways to store credentials or environment variables. As this step only deals with +public keys, one need not take extraordinary measures to prevent leakage. + +### Alternative 1: Set and use a secret + +```bash +fly secrets set "AUTHORIZED_KEYS=$(cat ~/.ssh/id_rsa.pub)" +``` + +Powershell users will want to use the following instead: + +```powershell +fly secrets set "AUTHORIZED_KEYS=$(Get-Content $HOME\.ssh\id_rsa.pub)" +``` + +Next, update your entrypoint script to contain the following: + +```bash +echo $AUTHORIZED_KEYS > /root/.ssh/authorized_keys +``` + +If you are running with sudo, you would want to do the following instead: + +```bash +echo $AUTHORIZED_KEYS | sudo tee /root/.ssh/authorized_keys > /dev/null +``` + +### Alternative 2: Copy from a volume + +If you have a volume and if you can arrange to upload the file +there, you can directly copy it as a part of your entrypoint script. + +```bash +cp /volume/.ssh/authorized_keys /root/.ssh +``` + +## [RECOMMENDED] Make SSH host keys stable + +The first time you SSH into a server you will be presented with a fingerprint for the server you are accessing. +If you accept that fingerprint, it will be added to your `known_hosts` file in your `.ssh` directory. This key +is generated when you install `openssh-server`. + +The issue arises when you redeploy your application. If something changes (or your docker cache expires), +the installation of `openssh-server` may be rerun, and new keys will be generated. To avoid these keys from +being used, you can capture and restore the keys. + +The following assumes that you have a [volume](../../apps/volume-storage/) mounted at `/volume`, and an entrypoint +script. + +```bash +mkdir -p /volume/.ssh + +if [[ "$(ls /volume/.ssh/*_key)" = "" ]]; then + cp /etc/ssh/*_key /volume/.ssh +else + cp /volume/.ssh/*_key /etc/ssh +fi +``` + +Notes: +* These keys are expected to be private, so if you go with an alternate route, make sure that these values are + not committed to a public repository unencrypted. +* If you are running with `sudo`, you need to add `sudo` before the `mkdir`, the `ls` and both of the `cp` statements. +* If you have multiple machines, you may want all of them to [share the same keys](https://security.stackexchange.com/a/89621+external). + +## [OPTIONAL] Configure client user and aliases + +Your full dnsname may be a mouthful, the user the application runs under may be different than the one you use on your laptop. +the port you expose may be non-standard, or you may have multiple machines and a desire to be able to SSH into a specific one. + +If any of these apply to you, you can create or update a file named [`config`](https://www.ssh.com/academy/ssh/config+external) in your `.ssh` directory. +Following is an example that illustrates addressing a number of the above cases: + +```config +Host appname + Hostname appname.fly.dev + User rails + Port 2222 +Host *.appname + HostName %h.internal + ProxyJump rails@appname.fly.dev:2222 + User rails + Port 2222 +``` + +Additionally, if you want to avoid the fingerprint checking, you can add the following to each of the `Host` entries: + +```config +StrictHostKeyChecking no +``` + +## Related reading + +- [`flyctl` SSH commands](https://fly.io/docs/flyctl/ssh/) Official reference for how to use `fly ssh`/`fly ssh sftp` to access running machines. +- [Multiple processes inside a Fly.io app](https://fly.io/docs/app-guides/multiple-processes/) Explains how to run multiple process groups in a single machine/container. Relevant when you’re adding `sshd` alongside your main app. \ No newline at end of file diff --git a/blueprints/per-user-dev-environments.html.md b/blueprints/per-user-dev-environments.html.md new file mode 100644 index 0000000000..5321ebc3aa --- /dev/null +++ b/blueprints/per-user-dev-environments.html.md @@ -0,0 +1,69 @@ +--- +title: Per-User Dev Environments with Fly Machines +layout: docs +nav: guides +date: 2025-04-02 +--- + +
+ Illustration by Annie Ruygt of different envrionments under glass domes +
+ +## Overview + +Fly Machines are fast-launching VMs behind [a simple API](https://fly.io/docs/machines/api), enabling you to launch tightly isolated app instances in milliseconds [all over the world](https://fly.io/docs/reference/regions/). + +One interesting use case: running isolated dev environments for your users (or robots). Fly Machines are a safe execution sandbox for even the sketchiest user-generated (or LLM-generated) code. + +This guide explains how to use Fly Machines to securely host ephemeral development and/or execution environments, complete with [dynamic subdomain routing](/docs/networking/dynamic-request-routing) using `fly-replay`. + +## What your architecture should include + +- **Router app(s)** + - A Fly.io app to handle requests to wildcard subdomains (`*.example.com`). Uses `fly-replay` headers to transparently redirect each request to the correct app and machine. If you have clusters of users (or robots) in different geographic regions, you can spin up a router app in multiple regions. See [Connecting to User Machines](/docs/blueprints/connecting-to-user-machines/) for details on how to implement the routing pattern. +- **User apps (pre-created)** + - Dedicated per-user (or per-robot) Fly apps ([more about why you should create a dedicated app per customer/robot](/docs/machines/guides-examples/one-app-per-user-why)), each containing isolated Fly Machines. App and Machine creation is not instantaneous, so we recommend provisioning a pool of these before you need them so you can quickly assign upon request. +- **Fly Machines (with optional volumes)** + - Fast-launching VMs that can be attached to persistent [Fly Volumes](/docs/volumes). + +### Example Architecture Diagram + +Diagram showing router app directing traffic to user apps containing Fly Machines with volumes + +### Router app(s) + +Your router app handles all incoming wildcard traffic. Its responsibility is simple: + +- Extract subdomains (like `alice.example.com` → `alice-123`). +- Look up the correct app (and optionally machine ID) for that user. +- Issue a `fly-replay` header directing the Fly Proxy to [internally redirect the request](/docs/blueprints/connecting-to-user-machines/#using-fly-replay) (this should add no more than ~10 milliseconds of latency if the router app is deployed close to the user). +- When appropriate, use [replay caching](/docs/networking/dynamic-request-routing/#replay-caching) to further reduce latency and load on the router app. +- Make sure you've added [a wildcard domain](/docs/networking/custom-domain/#get-certified) (*.example.com) to your router app (read more about the [certificate management endpoint here](/docs/networking/custom-domain-api/)). + +### User apps + +Creating apps dynamically for each user at request time can be slow. To ensure fast provisioning: + +- **Pre-create** a pool of Fly apps and machines ahead of time (using the [Fly Machines API or CLI](/docs/apps/overview/)). +- Store app details (e.g., app_name: `alice-123`) in a datastore accessible to your router app. +- Assign apps to users at provisioning time. + +**Fly Machines** + +You'll want to spin up at least one Machine per user app (but apps can have as many Machines as needed). If your dev environments need persistent storage (data that should survive Machine restarts): + +- Attach Fly Volumes to each machine at creation time. +- Keep in mind that machine restarts clear temporary filesystem state but preserve volume data. +- Learn more about the [Machines API resource](/docs/machines/api/machines-resource/) and the [Volumes API resource](/docs/machines/api/volumes-resource/). + +## Pointers & Footguns + +- **Machines & volumes are tied to physical hardware:** hardware failures can destroy machines and attached volumes. **Always persist important user data** (code, config, outputs) to external storage (like [Tigris Data](/docs/tigris/#main-content-start) or AWS S3). +- **Your users will break their environments:** pre-create standby machines to handle hardware & runtime failures, or the inevitable user or robot poisoned environment. Pre-create standby machines that you can quickly activate in these scenarios. +- **Machine restarts reset ephemeral filesystem:** the temporary Fly Machine filesystem state resets on Machine restarts, ensuring clean environments. However, volume data remains persistent, making it useful for retaining user progress or state. + +## Related reading + +- [Connecting to User Machines](/docs/blueprints/connecting-to-user-machines/) How to manage routing and networking when you spin up a fleet of machines dedicated to individual users. +- [Multi‑container Machines](/docs/machines/guides-examples/multi-container-machines/) When your per‑user environment needs sidecars (logs, metric exporters, agent processes) alongside the main service. + diff --git a/blueprints/private-applications-flycast.html.md b/blueprints/private-applications-flycast.html.md new file mode 100644 index 0000000000..932efdacda --- /dev/null +++ b/blueprints/private-applications-flycast.html.md @@ -0,0 +1,199 @@ +--- +title: Run private apps with Flycast +layout: docs +sitemap: true +nav: guides +author: xe +categories: + - networking +date: 2024-06-17 +--- + +

+ +## Overview + +A lot of the time your applications are made to be public and shared with the world. Sometimes you need to be more careful. When you deploy your apps on Fly.io, you get a private network for your organization. This lets your applications in other continents contact each other like they were in the same room. + +Sometimes you need a middle ground between fully public apps and fully private apps. [Flycast](/docs/networking/flycast/) addresses are private but global IPv6 addresses inside your private network that go through the Fly Proxy, so you get all of the load management and Machine waking powers that you get for free with public apps. + +This blueprint covers what Flycast is, when and why you’d want to use it, and shows you how to create an instance of Ollama that you can connect to over Flycast. + +## What is Flycast? + +Before we get started, let’s talk about Flycast and when you would want to use it. In general we can split every kind of Fly App into two categories: public apps and private apps. + +A public app is what you’d expose to the public Internet for your users. These are usually hardened apps that allow users to do some things, but have access limitations that prevent them from stepping outside their bounds. These are mostly programs that listen over HTTP for browsers to interact with. Your users connect to a public app through the platform router via the .fly.dev domain or whatever other domain you’ve set up. + +A private app is something internal, like a database server or a worker queue. These are things that run in the background and help you get things done, but are intentionally designed to NOT be exposed to the public Internet. You wouldn’t want to expose your Postgres or Valkey servers to anyone, would you? + +However, with a fully private app, all connections go directly to the Machines via their .internal addresses, so you have to keep them running 24/7 to maintain connectivity. This is fine for services like database engines where you want them to be running all the time, but what about an admin panel? You want your admin panel to be separate from your main app so that users can ever get into it, even by accident, but you also want it to shut down when it’s not in use. + +Flycast exists for this middle category of apps. With Flycast, your apps are only visible over your organization’s private network, but any traffic to them goes through the proxy so they can turn on when you need them and turn off when you don’t. This allows your administrative panels to be physically separate so that users can’t access them. + +When you want to connect to an app via Flycast, you connect to `appname.flycast`. + +### Security note + +Just a heads-up. In general, it’s a bad idea to assume that network access barriers like Flycast or NAT are security layers. At best, this is an obfuscation layer that makes it more difficult for attackers to get into private applications. Flycast is not a replacement for authentication in your private applications. With Flycast, you don’t know _who_ a request is coming from, but you do know that it’s coming from _something_ or _someone_ in your private network. + +One of the biggest platform features that uses Flycast out of the box is Fly Postgres. Even though Flycast addresses are local to your private network, Fly Postgres still configures usernames and passwords for your database. + +## Goal + +We'll show Flycast off by setting up an instance of [Ollama](https://ollama.com+external). + +Ollama is a program that wraps large language models and gives you an interface like Docker so that you can run open-weights large language models privately on your own device. Large language models are computationally expensive to run, so being able to offload them to a GPU-powered Fly Machine means you can hack all you want without burning up your precious battery life. + +Ollama doesn’t ship with authentication by default. When you create an instance of Ollama, anyone can access it without entering in a username, password, or API key. This is fine for running your models on your own computer; but it means that if you expose it to the internet, anyone can use it and run models whenever they want. This could rack up your bill infinitely. + +This is where Flycast comes in. Flycast lets you run a copy of Ollama on your private network so that you and your apps can access it, but nobody else. Flycast also lets you have the platform turn off your Ollama server when you’re not using it, which will save you money. This fits into that middle ground case that Flycast covers perfectly. + +## Prerequisites + +To get started, you need to do the following: + +- [sign up or sign in](/docs/getting-started/sign-up-sign-in/) to Fly.io +- [install flyctl](/docs/flyctl/install/) (the Fly CLI) + +If you want to interact with your Flycast apps—like an Ollama instance—from your computer, you’ll need to [jack into your private network with WireGuard](/docs/blueprints/connect-private-network-wireguard/). + +## Steps + +Create a new folder on your computer called `ollama`. This is where we’ll put the Ollama configuration. Open a terminal in that folder and run the fly launch command: + +``` +fly launch --from https://github.com/fly-apps/ollama-demo --no-deploy +``` + +This command creates a new fly app from the [`ollama-demo` template](https://github.com/fly-apps/ollama-demo+external) and tells the flyctl command to not deploy it after you create the app. If we don’t do this, then the platform will create public IPv4 and IPv6 addresses, which will make this a public app. The name you choose when you create your app will be used to connect to your app over Flycast. + +Next, allocate a Flycast address for your app with the `fly ips allocate-v6` command: + +``` +$ fly ips allocate-v6 --private +``` + +Now you can deploy the app with the `fly deploy` command: + +``` +$ fly deploy +``` + +After that finishes, you can see the list of IP addresses associated to an app with `fly ips list`: + +``` +$ fly ips list +VERSION IP TYPE REGION CREATED AT +v6 fdaa:3:9018:0:1::7 private global 23h12m ago + +Learn more about [Fly.io public, private, shared and dedicated IP addresses](/docs/reference/services/#ip-addresses). +``` + +This app only has one IP address: a private Flycast IPv6 address. If had public IP addresses, it'd look like this: + +``` +$ fly ips list -a recipeficator +VERSION IP TYPE REGION CREATED AT +v6 2a09:8280:1::37:7312:0 public (dedicated) global May 30 2024 13:51 +v4 66.241.124.113 public (shared) Jan 1 0001 00:00 +``` + +Now that we've proven it's private, let’s open an interactive shell Machine to play around with Flycast. Create the shell Machine with `fly machine run`: + +``` +$ fly machine run --shell ubuntu +root@e784127b51e083:/# +``` + +The Ubuntu image we chose is very minimal, so we need to install a few tools such as ping, curl, and dig: + +``` +# apt update && apt install -y curl iputils-ping dnsutils +``` + +My app is named`xe-ollama`, so let’s look up its `.flycast` address with `nslookup xe-ollama.flycast`: + +``` +# nslookup xe-ollama.flycast +Server: fdaa::3 +Address: fdaa::3#53 + +Name: xe-ollama.flycast +Address: fdaa:3:9018:0:1::7 +``` + +It matches that IP address from earlier. Now let’s see what happens when we ping it: + +``` +# ping xe-ollama.flycast -c2 +PING xe-ollama.flycast (fdaa:3:9018:0:1::7) 56 data bytes +64 bytes from fdaa:3:9018:0:1::7: icmp_seq=1 ttl=63 time=0.138 ms +64 bytes from fdaa:3:9018:0:1::7: icmp_seq=2 ttl=63 time=0.223 ms + +--- xe-ollama.flycast ping statistics --- +2 packets transmitted, 2 received, 0% packet loss, time 1009ms +rtt min/avg/max/mdev = 0.138/0.180/0.223/0.042 ms +``` + +Perfect, now let’s make a request to the Ollama app with curl: + +``` +# curl http://xe-ollama.flycast +``` + +It took a moment for Ollama to spin up, and now we get a happy “Ollama is running” message. Wait a few moments so your Ollama app goes to sleep and run the `time` command to see how long the first request takes: + +``` +# time curl http://xe-ollama.flycast +Ollama is running +real 0m9.144s +user 0m0.003s +sys 0m0.003s +``` + +It took a few seconds for the platform to wake up Ollama and make sure it was ready for your requests. The next request is a lot faster: + +``` +# time curl http://xe-ollama.flycast +Ollama is running +real 0m0.043s +user 0m0.003s +sys 0m0.003s +``` + +And if you wait a few moments, it’ll spin back down. + +### Running Llama 3 + +Now that we’ve set up Ollama and demonstrated the platform turning it off and on for you, let’s run Llama 3. Exit out of that shell Machine with control-D so we can make a new one with the Ollama client installed. + +Create an Ollama shell using `fly machine run`: + +``` +$ fly machine run --shell ollama/ollama +``` + +Once that starts up, point the Ollama client to your Flycast app by setting the `OLLAMA_HOST` environment variable: + +``` +# export OLLAMA_HOST=http://xe-ollama.flycast +``` + +Then you can ask Llama 3 anything you want: + +```javascript +# ollama run llama3 "Why is the sky blue?" +``` + +It took a moment for Ollama to get ready and download the image, then it downloaded it and answered your question. Once it’s been idle for a moment, the platform will turn Ollama back off. + +And there we go! We’ve covered what Flycast is, why you’d want to use it, and set up an instance of Ollama to show it off. + +## Related reading + +We've talked about Flycast in some past blog posts and blueprints: + +- [Autostart and autostop private apps](/docs/blueprints/autostart-internal-apps/) +- [Deploy Your Own (Not) Midjourney Bot on Fly GPUs](https://fly.io/blog/not-midjourney-bot/) +- [Scaling Large Language Models to zero with Ollama](https://fly.io/blog/scaling-llm-ollama/) diff --git a/blueprints/remote-mcp-servers.html.md b/blueprints/remote-mcp-servers.html.md new file mode 100644 index 0000000000..90c5f2b01c --- /dev/null +++ b/blueprints/remote-mcp-servers.html.md @@ -0,0 +1,75 @@ +--- +title: Deploying Remote MCP Servers +layout: docs +nav: guides +date: 2025-04-15 +--- + +
+ Illustration by Annie Ruygt of bird working at their computer while drinking coffee +
+ +## Overview + +The Model Context Protocol (MCP) is a fun new way to give LLMs new powers. Originally developed by Anthropic, the protocol has since been adopted by OpenAI (with Google Gemini support in the works at the time of writing). + +The protocol defines a standardized way of connecting tools and providing additional context to LLMs, not dissimilar to the way USB provides a standardized way to connect computers to peripherals and devices. Fly Machines are tightly isolated VMs that are perfect for running MCP servers. + +This guide will help you understand, at a very high level, how to build, deploy, and connect remote MCP servers on Fly.io. Keep in mind that this is an nascent, still-emerging protocol – details are subject to change. For specific implementation details, the [Model Context Protocol](https://modelcontextprotocol.io/) site is the authoritative source of truth (complete with [a handy .txt version for LLM prompting](https://modelcontextprotocol.io/llms-full.txt)). + +We've also started to integrate some MCP tooling into the `flyctl` that should give you a more concrete sense of how you might deploy MCP servers on Fly.io (read more in [this community forum post](https://community.fly.io/t/running-mcps-on-and-with-fly-io/24588)). Note that our MCP implementation is experimental and may still have sharp edges! + +## Remote MCP Servers + +MCP to date has been largely local-only, but the protocol also [specs out transports](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports) for remote MCP servers. Remote MCP servers solve a bunch of problems: + +- Easier to update a centralized server instead of dispersed, local packages +- You can give MCP clients persistent connections that don't evaporate when someone closes their laptop +- Securely sandbox MCP server activity in case the robots go rogue + +## Single Tenant or Multi-tenant + +There are broadly two patterns you might want to follow when deploying a remote MCP server to Fly.io: + +1. Multi-tenant MCP server (one app, many users) +2. Single-tenant MCP servers (one app per user) + +We're partial to the single-tenant pattern – it ensures proper isolation, and also helps with minimize your Fly.io bill: unused Machines can stop and start as needed, so you won't waste resources on idle users (more about why we think one app per customer is the right pattern [here](https://fly.io/docs/machines/guides-examples/one-app-per-user-why/)). `fly-replay` makes it easy to route requests to the correct app / Fly Machine (see more detail about this pattern in [Per-user Dev Environments with Fly Machines](https://fly.io/docs/blueprints/per-user-dev-environments/)). + +## Multi-tenant MCP Servers + +Diagram showing multi-tenant MCP server architecture on Fly.io + +You'll need two main components: + +1. **MCP-Remote Shim (optional)**: Tiny client-side proxy that connects local MCP clients to your remote servers (only needed if the MCP client doesn't support auth and / or streamable HTTP requests to a remote MCP server). Handles authentication via a secret shared between the shim and the router app (authentication could be a simple API token, username + password, or a full OAuth dance). Securely stores and refreshes tokens as needed. To see an example, try the experimental `fly mcp proxy` command in the `flyctl`. This sets up a local proxy that forwards MCP client requests to a remote URL (more details in [the docs](https://fly.io/docs/flyctl/mcp) and [the community forum](https://community.fly.io/t/running-mcps-on-and-with-fly-io/24588)). +2. **MCP Server App**: This runs the actual MCP goodness and authenticates requests from the MCP client. Should have a single streamable HTTP endpoint path for MCP client connections, as well as any specific business logic or other integrations. To try our implementation, include the experimental `fly mcp wrap` command in the Dockerfile for your MCP server app. This instantiates a lightweight HTTP server to receive requests forwarded from the local MCP client via `fly mcp proxy` (more details in [the docs](https://fly.io/docs/flyctl/mcp) and [the community forum](https://community.fly.io/t/running-mcps-on-and-with-fly-io/24588)). + +## Single-tenant MCP Servers + +Diagram showing single-tenant MCP server architecture on Fly.io + +There are three main components: + +1. **Router App**: Receives requests from the MCP client and handles auth, then routes requests to per-user apps + Fly Machines with `fly-replay`. See [Connecting to User Machines](/docs/blueprints/connecting-to-user-machines/) for details on implementing the routing pattern. Optionally handles user management and permissions. You can use the experimental `fly mcp wrap` command in the Dockerfile of your router app to instantiate a lightweight HTTP server to receive requests forwarded from a local MCP client (more details in [the docs](https://fly.io/docs/flyctl/mcp) and [the community forum](https://community.fly.io/t/running-mcps-on-and-with-fly-io/24588)). Note that `fly mcp wrap` does not handle request routing – you'll have to implement that separately. +2. **MCP Server Apps**: Per-user (or per-team) apps that run the actual MCP goodness. Should have a single streamable HTTP endpoint path for MCP client connections, as well as any specific business logic or other integrations. +3. **MCP-Remote Shim (optional)**: Tiny client-side proxy that connects local MCP clients to your remote servers (only needed if the MCP client doesn't support auth and / or streamable HTTP requests to a remote MCP server). Handles authentication via a secret shared between the shim and the router app (authentication could be a simple API token, username + password, or a full OAuth dance). Securely stores and refreshes tokens as needed. To see an example, try the experimental `fly mcp proxy` command in the `flyctl`, which sets up a local proxy that forwards MCP client requests to a remote URL (more details in [the docs](https://fly.io/docs/flyctl/mcp) and [the community forum](https://community.fly.io/t/running-mcps-on-and-with-fly-io/24588)). + +## How Users Experience It + +From your users' perspective, the experience should be delightfully simple: + +1. They add a new MCP server to their MCP client with a simple one-liner +2. If needed, the user is walked through an authentication flow (ie a browser window pops open to kick off the OAuth dance or provide an API key) +3. After logging in, the MCP client can now access all the tools you've exposed +4. The connection persists across restarts without re-authentication + +## Taking It Further + +If you'd like your MCP servers to connect to third party OAuth APIs without having to directly handle API tokens, consider using [ssokenizer](https://github.com/superfly/ssokenizer) (or [tokenizer](https://github.com/superfly/tokenizer) for generic, non-OAuth flows). + +## Related reading + +- [Model Context Protocol (MCP) Documentation](/docs/mcp/) Learn more in our reference for MCP: what it is, how it works, and how to deploy servers. +- [Connecting to User Machines](/docs/blueprints/connecting-to-user-machines/) Find out about patterns for routing from a central router app to per‑user apps/machines via `fly‑replay`, which is useful when you build single‑tenant MCP setups. +- [Launching MCP Servers with flyctl](/docs/flyctl/mcp-server/) Read about the experimental `fly mcp server` command: how to start a local or remote MCP server using `flyctl`, including flags and usage. diff --git a/blueprints/resilient-apps-multiple-machines.html.md b/blueprints/resilient-apps-multiple-machines.html.md new file mode 100644 index 0000000000..719e8f2bde --- /dev/null +++ b/blueprints/resilient-apps-multiple-machines.html.md @@ -0,0 +1,175 @@ +--- +title: Resilient apps use multiple Machines +layout: docs +nav: guides +date: 2025-09-12 +--- + +
+ Illustration by Annie Ruygt of an armoured truck with a Fly balloon logo +
+ +
+**Fly Machines are fast-launching VMs, the basic compute unit of the Fly.io platform. Each Machine runs on a single physical host. If that host fails, the Machine goes down and does not automatically start again on another host.** +
+ +## Overview + +To make your app resilient to single-host failure, create at least two Machines per app or process. Fly Proxy autostop/autostart and standby Machines are built-in platform features that you can use to start extra Machines only when needed. + +--- + +## Resiliency: Why and When + +Why resiliency? Machines are great until they aren’t. Hosts fail. Regions go dark. Networks get flaky. If your app only runs on one Machine, any of those failures means downtime. Running more than one Machine, or spreading them across regions, is how you keep the lights on when the infrastructure underneath you stumbles. + +### When to use it + +If you are building something people rely on, you probably want some resiliency. That includes production apps with paying users, customer-facing services where downtime makes people angry, or global apps that need to respond quickly no matter where users are. Regulations can also push you here if you are required to keep systems up in specific geographies. + +### When it might not be worth it + +For some apps, downtime just does not matter very much. A side project, a staging environment, or an internal tool might run fine on a single Machine. If the stakes are low, adding extra Machines can be more complexity than you need. + +### Performance + +Machines are really fast to start and stop. If your app has a lot of users, or bursts of high usage, then the Fly Proxy can load balance requests and automatically stop and start Machines based on traffic to your app. Keep one or more Machines running all the time if your app needs even faster first response times. + +### Costs + +You pay only for `rootfs` when Machines aren't running, not for CPU and RAM. Machine rootfs is [cheap](/docs/about/pricing/#stopped-fly-machines), like 18 cents a month for an average Elixir app that uses 1.2 GB of rootfs. + +--- + +## Multiple Machines for apps with services + +You can add more Machines for Fly Proxy to start and stop as needed, which is great for apps that have built-in replication or that don't share data. + +### You get two Machines on first deploy + +When you deploy an app for the first time with the `fly launch` or `fly deploy` command, you automatically get two identical Machines for processes that have HTTP/TCP services configured in `fly.toml`. The Machines have autostop/autostart enabled so that Fly Proxy can start and stop them based on traffic to your app. You'll also get this default redundant Machine when you `fly deploy` after scaling to zero. + +
+**Volumes:** You'll only get one Machine with `fly launch` for processes or apps with volumes mounted. Volumes don't automatically replicate your data for you, so you'll need to set that up before intentionally creating more Machines with volumes. +
+ +### Add more Machines yourself + +If your app doesn't already have multiple Machines with autostop/autostart configured, then you can set it up yourself. You can create any number of Machines to both meet user demand and provide redundancy against host failures. + +#### 1. Set up autostop/autostart + +Use [Fly Proxy autostop/autostart](/docs/launch/autostop-autostart/) to automatically stop and start Machines based on traffic. Keep one or more Machines running in your primary region if you want to. Example `fly.toml` config: + +```toml +[http_service] + internal_port = 8080 + force_https = true + auto_stop_machines = "stop" # Fly Proxy stops Machines based on traffic + auto_start_machines = true # Fly Proxy starts Machines based on traffic + min_machines_running = 0 # No. of Machines to keep running in primary region + [http_service.concurrency] + type = "requests" + soft_limit = 200 # Used by Fly Proxy to determine Machine excess capacity +``` + +Fly Proxy uses the concurrency `soft_limit` to determine if Machines have excess capacity. Learn more about [how Fly Proxy autostop/autostart works](/docs/reference/fly-proxy-autostop-autostart/). + +**Using the Machines API:** To add or change the autostop/autostart settings with the Machines API, use the settings in the `services` object of the [Machine config](/docs/machines/api/machines-resource/#machine-config-object-properties) in your `create` or `update` calls. + +#### 2. Create more Machines + +Create extra Machines, or even just one extra Machine, using flyctl: + +``` +fly scale count 2 +``` + +Or deploy even more Machines in more than one region: + +``` +fly scale count 20 --region ams,ewr,gig +``` + +Learn more about [scaling the number of Machines](/docs/apps/scale-count/). + +**Using the Machines API:** To scale the number of Machines with the Machines API, you can "clone" Machines. Get and modify a Machine config and use that config to create new Machines in any region. See the[ Machines resource docs](/docs/machines/api/machines-resource/). + +--- + +## Standby Machines for apps without services + +When apps or processes are running tools like cron that don't require local storage or accept external requests, it's common to run only one Machine. Since these Machines don't have services configured, they can't be automatically started and stopped by the Fly Proxy. To add redundancy against host failures for this kind of Machine, use a standby Machine; it stays stopped and ready to take over in case the original Machine becomes unavailable. + +Unlike Fly Proxy autostop/autostart, which starts Machines based on app traffic, a standby Machine watches the Machine it's paired to and starts only if that Machine becomes unavailable. Learn more about [standby Machines](/docs/reference/app-availability/#standby-machines-for-process-groups-without-services). + +### You get a standby Machine on first deploy + +When you deploy an app for the first time with the `fly launch` or `fly deploy` command, you automatically get a standby Machine for processes that don't have services configured in `fly.toml` (and therefore aren't visible to Fly Proxy). That standby Machine is a copy of the original, running Machine. You'll also get this standby Machine when you `fly deploy` after scaling to zero. + +### Create a standby Machine yourself + +If you don't already have a standby Machine for your worker Machines, then you can create one yourself. + +To create a standby Machine that will take over the same tasks when a worker Machine is down, use this one command to clone the worker Machine and make the clone a standby: + +``` +fly machine clone --standby-for +``` + +**Using the Machines API:** To create a standby Machine with the Machines API, get and modify your worker Machine config and use that config to create a new standby Machine. Specify the standby in the Machine's config object: [config.standbys](/docs/machines/api-machines-resource/#machine-config-object-properties). + +### Bonus uses for standby Machines + +A standby Machine doesn't need to be a copy of the Machine it's watching. You can configure the standby Machine to run with any image you like when a watched Machine becomes unavailable; you can use it to run a recovery script, trigger alerts, or anything else. You can create a standby Machine for any Machine, including one with services. + +To create a standby Machine from a different image: + +``` +fly machine run --standby-for +``` + +--- + +## Scaling to multiple regions + +Running multiple Machines in one region protects you from host failures. But entire regions can fail too. Power cuts, network issues, and upstream outages do not happen often, but when they do, every Machine in that region goes down. If your app needs to weather that kind of failure, the next step is spreading out across regions. + +### When to use it + +Multi-region setups make sense when uptime really matters. That might mean production systems with strict SLAs, customer-facing apps where downtime causes pain, or global services that need to stay responsive close to users. Regulations can also be a factor if you need to keep infrastructure running in specific geographies. + +### Example + +You can place Machines in more than one region with a single command: + +``` +fly scale count 6 --region ams,ewr,syd +``` + +This example runs two Machines in Amsterdam, two in New Jersey, and two in Sydney. Fly Proxy will route users to the closest healthy region. + +One important caveat: volumes do not replicate across regions. If your app writes data, you need to think carefully about where that data lives and how it syncs. A common approach is to keep one primary “write” region and treat others as read-only. + +### Keep in mind + +Multi-region comes with real advantages, but also some trade-offs. Users near your regional Machines will see faster responses, but coordinating data across regions is hard, especially since most apps are not designed for global writes. And while redundancy improves reliability, it also increases your costs: more Machines in more places means more expense, and cross-region traffic can add up quickly. + +--- + +## Summing up + +Resiliency builds in layers. At the base, you have a single Machine: cheap, simple, and fragile. Add a second Machine in the same region and you are protected from host failures. Spread Machines across regions and you are resilient to outages that take down an entire site, with the added bonus of faster responses for users around the world. + +Each layer improves availability, but it also adds cost and complexity. The right choice depends on what you are building and who is depending on it. + +Whatever strategy you choose, practice it. Do not wait for a real outage to find out how your app behaves. Stop a Machine and see if traffic shifts. Kill a region and watch how your data layer responds. A resiliency plan only works if you have rehearsed it enough to trust it. + +## Related reading + +If you want to dig deeper into how Fly apps stay available and what failure modes to expect, these are good next steps: + +- [App availability and resiliency](/docs/apps/app-availability/): An overview of the all features that can make your app more resistant to events like hardware failures or outages. +- [Multi-region with Fly-Replay](/docs/blueprints/multi-region-fly-replay/): How to serve reads close to users and route writes back to a primary region. +- [Machine placement and regional capacity](/docs/machines/guides-examples/machine-placement/): How Fly decides where to place your Machines, how to guide placement yourself, and what happens when a region runs out of capacity. +- [Run Ordinary Rails Apps Globally](/ruby-dispatch/run-ordinary-rails-apps-globally/): Project-level example of Rails distributed across regions with smart routing. diff --git a/blueprints/review-apps-guide.html.md b/blueprints/review-apps-guide.html.md new file mode 100644 index 0000000000..d59e192be3 --- /dev/null +++ b/blueprints/review-apps-guide.html.md @@ -0,0 +1,175 @@ +--- +title: "Git Branch Preview Environments on Github" +layout: docs +nav: guides +categories: + - ci + - github + - guide + - review apps + - cd +redirect_from: /docs/app-guides/review-apps-guide/ +--- + +
+ Illustration by Annie Ruygt of Github Octocat and Frankie with some apps +
+ +# Overview + +Review apps are a great way to preview new features, changes, and bug fixes. This guide will teach you how to automatically generate ephemeral "review apps" on Fly.io for each pull request (PR) using GitHub Actions. This approach can be applied to other Git services and source code versioning software that supports branching and hooks. + +## Quick Start + +If you've worked with GitHub Actions before and want to jump right in, here are the steps to take: + +1. Create a new repository secret on your GitHub repository called `FLY_API_TOKEN` and provide it with the returned value of `fly tokens org ` +1. Create a new workflow in your project at `.github/workflows/fly-review.yml` and add the following [YAML code](https://gist.github.com/anniebabannie/3cb800f2a890a6f3ed3167c09a0234dd). +1. Commit and push the changes containing your GitHub workflow. + +And that's it! Any new pull requests will trigger the creation of review apps for your application. + +By default, review apps will be available at `https://pr---.fly.dev`. + +The workflow described above will spin up a **single application** without other resources; if data stores, multiple Fly Apps, or other resources are required, see [Customizing your workflow](#customize-your-workflow). + + +For more details about each step, follow the guide below. + +## Setting up review apps on Fly.io + +### Set your Fly API token + +Your GitHub Action will need a Fly API token to deploy and manage new review apps on Fly.io. You can get this token using the following [flyctl](/docs/flyctl/install/) command: + +```cmd +fly tokens org +``` + +This token is specific to a single Fly.io organization and is valid for applications belonging the that organization. + +Next, let's save our API token as a secret in our GitHub repository. Visit the Settings tab in your repository, then Secrets and variables → Actions. Alternatively, you can visit `https://github.com///settings/secrets/actions`. There, create a new **Repository Secret** called `FLY_API_TOKEN` with the value of your token. + +### Create your GitHub Action + +Next, let's create a new GitHub Action by creating a new YAML file in your app's repository for our workflow: + +```cmd +touch .github/workflows/fly-review.yml +``` + +Then add the following to your new `fly-review.yml` file: + +```yaml +name: Deploy Review App +on: + # Run this workflow on every PR event. Existing review apps will be updated when the PR is updated. + pull_request: + types: [opened, reopened, synchronize, closed] + +env: + FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} + # Set these to your Fly.io organization and preferred region. + FLY_REGION: iad + FLY_ORG: personal + +jobs: + review_app: + runs-on: ubuntu-latest + outputs: + url: ${{ steps.deploy.outputs.url }} + # Only run one deployment at a time per PR. + concurrency: + group: pr-${{ github.event.number }} + + # Deploying apps with this "review" environment allows the URL for the app to be displayed in the PR UI. + # Feel free to change the name of this environment. + environment: + name: review + # The script in the `deploy` sets the URL output for each review app. + url: ${{ steps.deploy.outputs.url }} + steps: + - name: Get code + uses: actions/checkout@v4 + + - name: Deploy PR app to Fly.io + id: deploy + uses: superfly/fly-pr-review-apps@1.2.1 +``` + +In the `deploy` step, `superfly/fly-pr-review` contains the custom Action with a shell script that creates and deploys your review apps. If you're curious, check out [entrypoint.sh](https://github.com/superfly/fly-pr-review-apps/blob/main/entrypoint.sh) for full details on how the script works. + +### Commit and push your workflow + +Finally, commit and push your changes to your GitHub repository. Once pushed, any new PRs will spin up a review app for you to preview your code on Fly.io. + +## Customize your workflow + +Often our applications do not operate in isolation and require other resources to function. The sample workflow mentioned above spins up a stand-alone Fly App, but you can customize your `fly-review.yml` GitHub workflow to create additional Fly Apps and resources, such as a Redis, Memcached, etc. You can find examples of these in the [README](https://github.com/superfly/fly-pr-review-apps/tree/main) for `superfly/fly-pr-review-apps`. + +### Customize with inputs + +The `superfly/fly-pr-review-apps` custom GitHub Action also provides a number of **inputs** that can be added to tweak behavior. See the [README](https://github.com/superfly/fly-pr-review-apps/tree/main) for the complete list. + +These inputs can be added to the `deploy` step in your workflow using the `with` key, like so: + +```yaml +# fly-review.yml +# ... + steps: + # ... + - name: Deploy PR app to Fly.io + id: deploy + uses: superfly/fly-pr-review-apps@1.2.0 + with: + name: my-app-name-pr-${{ github.event.number }} + config: fly.review.toml +``` + +### Defining resource specs + +You probably won't need the same specifications for the Fly Machines used by review apps as you do for your production app. There are two ways to override the resources used in review apps. + +**1. Specifying a different `config` file:** By default, review apps will use the `fly.toml` in your application root, but you can create a separate TOML file with different specs using the `config` input, like so: + +```yaml +# fly-review.yml +# ... + steps: + # ... + - name: Deploy PR app to Fly.io + id: deploy + uses: superfly/fly-pr-review-apps@1.2.0 + with: + config: fly.review.toml +``` + +**2. Setting specs in GitHub Action inputs:** Alternatively, you can set the specs for the Fly Machines used by your review apps using the following inputs: + +- `vmsize` +- `cpu` +- `cpukind` +- `memory` +- `ha` + +### Default secrets and environment variables + +We recommend setting default values for any secrets or environment variables your review apps require so you don't have to set them individually. You can do this by defining each secret as either a **repository secret** _or_ an **environment secret** in your GitHub repo. If using environment secrets, these should be set on the environment used in your workflow, such as `review`. + +Once you have your secrets and environment variables set in GitHub, add each of them to your `fly-review.yml` workflow, separated by a space, like so: + +```yaml +# fly-review.yml +# ... + steps: + # ... + - name: Deploy PR app to Fly.io + id: deploy + uses: superfly/fly-pr-review-apps@1.2.0 + with: + secrets: FOOBAR=${{ secrets.FOOBAR }} SOME_API_KEY=${{ secrets.SOME_API_KEY}} +``` +## Related Reading + +- [Continuous Deployment with Fly.io and GitHub Actions](/docs/launch/continuous-deployment-with-github-actions/) Shows how to set up automated CI/CD pipelines using GitHub Actions and Fly.io — a natural complement to per‑PR review deployments. +- [Get information about an app](/docs/apps/info/) A handy guide to inspecting apps, machines, statuses via CLI/API — useful when you’re creating and destroying review apps on pull requests. \ No newline at end of file diff --git a/blueprints/rollback-guide.html.md b/blueprints/rollback-guide.html.md new file mode 100644 index 0000000000..89ea2ab437 --- /dev/null +++ b/blueprints/rollback-guide.html.md @@ -0,0 +1,134 @@ +--- +title: Rollback Guide +layout: docs +nav: guides +author: kcmartin +date: 2025-08-22 +--- + +
+ Illustration by Annie Ruygt of a Frankie the hot air balloon helping his developer friend roll back a bad deploy +
+ +## Overview: Why rollback? + +Fly.io runs your apps close to your users, in VMs you can spin up and tear down in seconds. But what if you spin up something broken? If a deploy goes wrong, it’s easy to roll back: just tell Fly to redeploy a previous image. There’s no special `rollback` command because you don’t need one. Rollbacks use the same deploy mechanism you already know. + +If you know which image was working before, you can deploy it again. You just need to know where to find the image, and how to tell Fly to use it. The trick is: you’re rolling back the _VM image_, not the database. If you ran migrations in that bad deploy, you’ll have to undo those separately in your application or migration code. Fly isn’t going to time-travel your data. + +--- + +## How to rollback + +### Step 1: See what you’ve shipped + +Fly keeps a history of your releases, including the exact container image each one used. Run: + +```bash +fly releases --app myapp --image +``` + +You’ll see a list of past releases with timestamps and image tags: + +``` +v42 2025-08-12T16:21Z registry.fly.io/myapp:deployment-01HXXXXX +v41 2025-08-10T09:14Z registry.fly.io/myapp:deployment-01HYYYYY +``` + +Pick the one before things went sideways. + +--- + +### Step 2: Deploy that exact image + +Let’s say `v41` was fine. You can redeploy its image directly: + +```bash +fly deploy --image registry.fly.io/myapp:deployment-01HYYYYY +``` + +That’s it — no rebuild, no code checkout, just telling Fly to boot the older VM image. This works as long as the image is still in the Fly registry. + +--- + +### Tip: Stop living in `deployment-01HXXXXX` hell: use human-readable tags + +The default auto-generated tags work for machines, not humans. You can make your life easier by tagging builds with something you’ll recognize later — a git commit hash, a version number, or even just `before-i-broke-prod`. + +When you deploy, pass `--image-label` to set a tag: + +```bash +fly deploy --image-label v1.2.3 +``` + +That command: + +1. Builds your app +1. Tags the image as `v1.2.3` +1. Deploys it + +Rolling back later becomes: + +```bash +fly deploy --image registry.fly.io/myapp:v1.2.3 +``` + +--- + +## How long does Fly keep your images? + +Right now, Fly doesn’t promise to store app images forever. If an image hasn’t been deployed for a while, it might be pruned from our registry to free up space. If you need a “guaranteed forever” rollback target, push your image to a public container registry (like Docker Hub, GHCR, or ECR) and deploy from there. See [Using the Fly Docker Registry](https://fly.io/docs/blueprints/using-the-fly-docker-registry/#to-deploy-a-public-image-from-docker-hub-or-another-public-registry) for details. + +--- + +## Things to watch out for + +Rollbacks are simple in Fly, but they aren’t magic. A few details are worth keeping in mind: + +- **Database schema drift** → If a deploy ran destructive migrations (dropping a column, renaming a table), rolling back the app image might not help. Plan migrations so they’re safe both forward and backward. +- **Image retention** → Images don’t live forever in Fly’s registry. For long-term rollback insurance, push builds to your own container registry. +- **Config, Secrets and `fly.toml`** → Rollbacks don’t undo config changes. When you deploy an older image, Fly still uses your current `fly.toml`, along with whatever env vars and secrets are set now. If a past deploy depended on an older config, you’ll need to update those settings manually when rolling back. +- **Autoscaling** → If your app scaled up during the bad release (say traffic spiked and Fly added more machines), those extra machines don’t vanish the instant you roll back. They’ll hang around until autoscaling decides to scale them down. This is expected behavior and your rollback still worked; you just have more machines than usual for a while. Read more about [metrics-based autoscaling](https://fly.io/docs/launch/autoscale-by-metric/). +- **Monitoring** → Always confirm a rollback stabilized things with logs , probes (`fly logs`, `fly status,` Grafana metrics), not just the success message from `fly deploy`. +- **Regions** → By default, rollbacks redeploy globally. If you run in multiple regions (say `iad`, `lhr`, `syd`), you might test a rollback in just one region before rolling back everywhere. You can do this with the `--region` flag: + +```bash +fly deploy --image registry.fly.io/myapp:v1.2.3 --region iad +``` +--- + +## Tip: Faster rollbacks with deploy strategies + +By default, Fly uses `rolling` deploys: new machines come up before old ones shut down. That’s safe for upgrades, but if you know the current release is bad, you may want a quicker cutover. + +### All-at-once rollback: + +```bash +fly deploy --image registry.fly.io/myapp:v1.2.3 --strategy immediate +``` + +This stops old machines and boots the rollback image as quickly as possible. + + + +### Controlled speedup: + +```bash +fly deploy --image registry.fly.io/myapp:v1.2.3 --max-unavailable 0.5 +``` + +This lets you take down up to half your machines at once, moving faster than a cautious rolling deploy while avoiding a full stop. + +Use `--strategy immediate` when speed matters more than downtime, and `--max-unavailable` when you want a hedge between safety and speed. + +--- + +## Summing up + +Rolling back on Fly is basically just _deploying an older image_. Once you’ve got the muscle memory for `fly releases --image` and `fly deploy --image`, it’s a quick, predictable way to recover from bad deploys. And if you start tagging images with human-readable labels, you’ll have an easier time remembering which one to trust when production is on fire. + +## Related reading + +- [Seamless Deployments on Fly.io](/docs/blueprints/seamless-deployments/) How to set up deploys that avoid downtime and make rollbacks less painful. +- [Using the Fly Docker Registry](/docs/blueprints/using-the-fly-docker-registry/) Details on image tagging and retention — key context for understanding how rollbacks work. +- [Staging and Production Isolation](/docs/blueprints/staging-prod-isolation/) Tips on environment isolation that reduce the blast radius of a bad release. \ No newline at end of file diff --git a/blueprints/seamless-deployments.html.md b/blueprints/seamless-deployments.html.md new file mode 100644 index 0000000000..2c15d12eb1 --- /dev/null +++ b/blueprints/seamless-deployments.html.md @@ -0,0 +1,108 @@ +--- +title: Seamless Deployments on Fly.io +layout: docs +nav: guides +author: kcmartin +date: 2025-09-23 +--- + +
+ Illustration by Annie Ruygt of a balloon doing a health check +
+ +## Overview + +
+**Zero-downtime deploys aren’t magic. They’re just health checks that work. Fly.io apps run on individual VMs, and we won’t start routing traffic to a new one until it proves it’s alive. Here's how to get seamless deploys without breaking things or relying on hope as a strategy.** +
+ +## Your deploy is only as good as your health checks + +Fly.io deployments look like magic when health checks are working. New Machines boot. Old ones go down. Users never notice. But behind the scenes, the [Fly Proxy](/docs/reference/fly-proxy/) is watching your Machines and only sending them traffic if they pass your health checks. + +[Health checks](/docs/reference/health-checks/) are how you tell the platform that your app is alive. If they're missing or broken, you're flying in the dark. + +In your `fly.toml`, you'll usually define health checks under `http_service.checks` (or the alternative `services.http_checks` if you're not using the http_service shortcut) for HTTP-based apps. These checks are simple: make a request to a path like `/`, expect a `200 OK` response, and fail if you get a redirect or timeout. If your app redirects HTTP to HTTPS, add a header like `X-Forwarded-Proto = "https"` to avoid false failures. + +The most common health check failure modes are easy to trip over: + +- getting 301/302s instead of 200 responses +- hitting a path that loads slowly under cold start conditions +- forgetting to delay checks with a `grace_period` after boot + +A typical HTTP health check might look like this: + +``` +[http_service.checks] + interval = "15s" + timeout = "2s" + grace_period = "10s" + method = "GET" + path = "/healthz" + headers = { X-Forwarded-Proto = "https" } +``` + +For apps that aren't serving HTTP, or where you want lighter checks, [TCP health checks](/docs/reference/health-checks/#service-level-checks) exist too. These just attempt to open a socket to the port you've configured; if the connection is accepted, the Machine is healthy. + +[Top-level health checks](/docs/reference/health-checks/#top-level-checks)—defined in the `[checks]` section of your `fly.toml`—are for observability, not traffic control. They don't influence routing; if a Machine fails one, we won’t pull it out of rotation. That makes them ideal for internal monitoring and alerting, especially for background workers or other non-public-facing services where you still want to know if something’s gone sideways, but don’t need Fly to reroute traffic automatically. + +Deployments get another layer: `machine_checks`. These aren't about routing traffic; they're about catching bad deploys before they go live. A `machine_check` boots a throwaway Machine, sets `FLY_TEST_MACHINE_IP`, and lets you poke it however you want. You can run integration tests, perform simulated traffic, check database access, or validate config. Fail the check, and the deploy halts. This is perfect for canary testing new releases. + +**Example:** + +``` +[services.machine_checks] + command = ["/bin/bash", "-c", "curl -f http://$FLY_TEST_MACHINE_IP:8080/healthz"] + interval = "30s" + timeout = "5s" +``` + +Note: `machine_checks` run in a temporary Machine with the new image before it's deployed anywhere else. They're isolated, so they won't affect your running app—but they can save you from pushing broken code into production. + +If a Machine fails a health check, Fly Proxy pulls it out of the rotation. That Machine can still be running, logging, and debugging, but it's invisible to your users. + +## Deployment strategies: choose your own trade-offs + +Working with health checks and at least two Machines, you can avoid downtime. The platform won't kill a healthy Machine until a new one is up and running. + +Fly.io supports a few deployment strategies in the `[deploy]` section of `fly.toml`: + +- **Rolling (default):** Replace Machines one at a time. Use `max_unavailable` to control how many go down at once (e.g., `1` or `25%`). +- **Immediate:** Burn the ships. All Machines update at once, no health checks, hope for the best. +- **Canary:** Start with one Machine. If it's healthy, continue with rolling. This can't be used with attached volumes. +- **Bluegreen:** Boot new Machines alongside old ones. Only switch traffic after all new Machines pass checks. Fastest, safest, but also can't use attached volumes. + +Every deploy also includes a smoke check: Fly watches Machines for \~10 seconds after they start. If they crash repeatedly, the deploy fails. + +Don't forget to set `wait_timeout` if your image is big or startup is slow. It's easy to hit timeouts before Machines even start. + +## One-off tasks with `release_command` + +Sometimes you need to run a script before your app is actually deployed, like a database migration or other one-off prep work. The `release_command` in your `fly.toml` lets you do exactly that. It spins up a fresh temporary Machine, runs your command, then shuts it down and destroys it. It doesn’t join your deployed Machines. + +This runs once per deploy attempt, using the built image and your app’s environment. It won’t have volumes attached, so it’s not for anything that needs persistent state. + +If `release_command` fails, the deploy won’t proceed. This is usually what you want. If the migration didn’t work, it's better to stop there. + +We’ve got more details, including how `ENTRYPOINT` and `CMD` are handled, in the [reference for `fly.toml`](/docs/reference/configuration/#run-one-off-commands-before-releasing-a-deployment). + +## Zero-downtime is a shared responsibility + +The Fly.io platform will keep your Machines healthy and traffic flowing. But there's one thing it can't fix: application incompatibility during deploys. + +Say you're changing your database schema. You run the migration in a `release_command`, then your Machines update one by one. If the old app code can't handle the new schema, users will see errors while old Machines are still running. Same goes for bluegreen deployments: the new Machines are healthy, but traffic doesn't switch until _all_ of them are. + +Fixing this is on you. You need to design schema and app changes that can coexist briefly. That usually means: + +- App code that works with both the old and new database schemas +- Migrations that add columns or tables but don't drop or rename until later + +Every framework has guidance for this. Read it. Follow it. Then health checks, deployment strategies, and the platform can do their jobs. + +Deploying without downtime isn't automatic. But with a little help from your health checks and some care in your app logic, it's well within reach. + +## Related reading + +- [Health Checks](/docs/reference/health-checks/) +- [Deploy an App](/docs/launch/deploy/) +- [App Configuration (`fly.toml`)](https://fly.io/docs/reference/configuration/) \ No newline at end of file diff --git a/blueprints/setting-concurrency-limits.html.md b/blueprints/setting-concurrency-limits.html.md new file mode 100644 index 0000000000..4f895904f3 --- /dev/null +++ b/blueprints/setting-concurrency-limits.html.md @@ -0,0 +1,48 @@ +--- +title: Setting Hard and Soft Concurrency Limits on Fly.io +layout: docs +nav: guides +author: kcmartin +date: 2025-05-15 +--- + +
+ Illustration by Annie Ruygt of Frankie the hot air balloon with two other balloon friends of different colors +
+ +## Overview + +On Fly.io, machines aren’t created automatically based on traffic; you create them manually. But once a machine is created, it can be stopped when idle (meaning you’re not paying for it), and started again instantly when load picks up. Our “autostart/autostop” feature can drive this process for you — but it only works well if you’ve tuned `soft_limit` and `hard_limit` correctly. Those limits tell Fly when a machine is too busy and it's time to bring more machines online. + +## How to tune concurrency limits + +Soft and hard limits for Fly Machines don’t have to be a black art. Here’s a quick rundown: + +- **hard_limit**: max concurrent requests the machine will ever handle. +- **soft_limit**: how many it can handle _comfortably_, before Fly begins starting more machines. Incoming requests are distributed on a round-robin fashion to started machines. + +The best way to dial this in is to actually test it. Based on the target app you’re working to scale, create a separate Fly app copy with the same machine config. Now set both limits absurdly high (say, 1000), and hammer it with load using something like `wrk`, `hey`, `ab`, `Locust`, or `k6`. Watch CPU, memory, and latency. + +What you’re looking for is the point where responses start to slow down below your comfort threshold. That’s your hard limit—just below that threshold. Your comfort level is up to you; a real-time application will want to keep responses under 100ms, but an ordinary CRUD application might be just fine with request times around 250ms. + +Now set your soft limit. The soft limit is there to give our proxy time to bring another machine online before your users notice lag. Example: if things start dragging at 20 concurrent requests, set `hard_limit`: 18 and `soft_limit`: 12. + +Once you’ve got your limits dialed in, turning on `auto_start_machines` and `auto_stop_machines` gives you hands-off scaling behavior that still respects the boundaries you’ve set. Your app shouldn’t need babysitting to handle spikes, and you won’t end up running more machines than you meant to. It’s a good balance between elasticity and control. + +## Capacity Planning: A Real Example + +Let’s say: + +- Each request takes ~100ms to serve. +- You’re expecting 750 requests per second. +- One machine handles about 10 concurrent requests before slowing down. + +That means you need 75 concurrent request slots to keep up. So: 750 rps * 100ms = 75 concurrent requests. At 10 per machine, you’ll need 8 machines. + +In order to serve requests as they arrive (that is, dispatch 750 requests/second) with an average processing time per request of 100ms, 75 “workers”, or concurrent requests handled, are needed. Based on the above machine capacity of 10 requests, 8 machines should be able to handle the proposed load. + +## Related reading + +- [Guidelines for Concurrency Settings](/docs/apps/concurrency/) Read more in this reference for the `soft_limit`, `hard_limit`, and `type` settings you’ll use in your `fly.toml`. +- [App Configuration: `http_service.concurrency`](/docs/reference/configuration/#http_service-concurrency) Check out this section of the `fly.toml` config reference explaining concurrency parameters. +- [Load Balancing on Fly.io](/docs/reference/load-balancing/) Find out how your concurrency limits affect how the Fly Proxy routes traffic across machines—and why your soft/hard limits matter. \ No newline at end of file diff --git a/blueprints/shared-nothing.html.markerb b/blueprints/shared-nothing.html.markerb new file mode 100644 index 0000000000..c4b192029b --- /dev/null +++ b/blueprints/shared-nothing.html.markerb @@ -0,0 +1,456 @@ +--- +title: "Shared Nothing Architecture" +subtitle: "or, Writing This App Was Easier Than Cloning Myself" +description: Comprehensive Overview of a Shared Nothing Application Deployed on Fly.io +cover: dancing-clones-cover.webp +thumbnail: dancing-clones-thumb.webp +artist: Annie Ruygt +artist_link: https://annieruygtillustration.com/ +alt: A number of couples dancing, including a two clones dancing with separate partners. +layout: docs +nav: guides +redirect_from: "/blog/shared-nothing/" +date: 2024-03-05 +author: rubys +--- + +
+ Illustration by Annie Ruygt of a number of couples dancing, including a two clones dancing with separate partners. +
+ +

+ +
+This guide from Rails and JS expert Sam Ruby is based on his personal capacity as a ballroom dance nerd. It covers how a serverful, stateful, shared-nothing architecture hit the spot for him when it came time to scale to multiple customers in different parts of the world. +
+ +## Introduction + +Once upon a time, I found myself double booked at a ballroom dance competition. Read that [story in more detail on my blog](http://intertwingly.net/blog/2022/08/13/Unretiring), but essentially: you’re only supposed to get one shot to dance in a given heat, and I ended up listed twice in the spreadsheet for that heat. I decided to write an app to take the load of manual entry and scheduling off of organizers, and so that no one would ever be asked to do something impossible and be at two places at one time. + +Showcase (that’s what I called it) started out as a Ruby on Rails app and a database; it kept track of competitors and their schedules for a single Showcase (that’s what these ballroom dance events are called). I ran it on a single Mac Mini in my home. + +Over time, it grew out of my attic and onto VMs in the cloud. It supports more events in different cities, and does more things. Conventional wisdom is that as you grow, you make each type of service reusable and put it into a separate set of machines. + +If this approach isn’t the best one for your app, throw it out the window! I did. In my case, it made a lot of sense to put all the services into a single VM, and scale out by repeating that set into new VMs. This works great for me. Maybe it’s right for your app, too! Maybe not! Prepare to bathe in the gory details of my stateful, serverful, shared-nothing monolith. + +## Shared-nothing architecures + +

+A shared-nothing architecture is a distributed computing architecture in which each update request is satisfied by a single node (processor/memory/storage unit) in a computer cluster. +

+ +As you might expect, a Showcase competition happens live and in a single location. Very few people need access to the system, and they’re often all in the same room; picture a few judges entering scores at the end of a number—that’s your peak simultaneous users. Data’s not shared between events–they don’t want or need to share a database and Redis cache. + +Shared-nothing architectures are nothing new. An excerpt from [Wikipedia](https://en.wikipedia.org/wiki/Shared-nothing_architecture): + +> The intent is to eliminate contention among nodes. Nodes do not share (independently access) the same memory or storage. One alternative architecture is shared everything, in which requests are satisfied by arbitrary combinations of nodes. This may introduce contention, as multiple nodes may seek to update the same data at the same time. + +While serverless is one kind of a shared-nothing architecture - this post explores another variation. One where servers are explicitly deployed, +users are assigned to servers, and users have databases that aren't shared. + +This application started, like most applications, small. And that's where the story begins. + +## A single event + +

+A Fly.io machine is not merely a Docker container, but rather a [full VM](https://fly.io/blog/docker-without-docker/). Treat it as such: run as many services as you like on a single machine. +

+ +My first showcase was in Harrisburg, Pennsylvania. The app I developed initially supported only that one event, and all of the services needed to support the event ran on a single machine. Like most Rails applications, +this consisted of a Rails application, a database, and (eventually) Redis. Over time a number of supporting services were added (NGINX, opensshd, rsyncd, and a small log "heartbeat" script). +Today, these all happily run on a single Fly.io machine. + +Conventional wisdom is that as you grow you put each type of service into a separate set of machines. That isn't always the right approach. + +An excerpt from DHH's [Majestic Monolith](https://signalvnoise.com/svn3/the-majestic-monolith/): + +> Every time you extract a collaboration between objects to a collaboration between systems, you’re accepting a world of hurt with a myriad of liabilities and failure states. What to do when services are down, how to migrate in concert, and all the pain of running many services in the first place. + +Another way to look at this is: as needs grow, do you really want to share a database and Redis instances between customers? (Or, in this case, between events?) +If "no" turns out to be a viable answer: go with it. Put all of the services needed to support one customer in one Docker image and then repeat that set as often +as needed. Eliminating the need for a network in producing a response to a request both increases throughput and reliability. + +There are a lot of [options for running multiple processes inside a Fly.io App](https://fly.io/docs/app-guides/multiple-processes/). +I use a combination of [ENTRYPOINT](https://github.com/rubys/showcase/blob/main/bin/docker-entrypoint) running a +[deploy](https://github.com/rubys/showcase/blob/main/bin/deploy) script and a [procfile](https://github.com/rubys/showcase/blob/main/Procfile.fly). + +Given the load for each event is only for one customer, I only need 1 CPU per server. While CPU is not a concern, it still is important to size +the amount of RAM needed. +Even with all of the services running, according to [Graftana/Fly Metrics](https://fly-metrics.net/), each instance uses about 220MB of RAM when idle, +and grows to 450MB when active. That being said, I have seen machines fail due to [OOM](https://en.wikipedia.org/wiki/Out_of_memory) with only 512MB, so I give +each machine 1GB. I also define an additional 2G of [swap](https://fly.io/docs/reference/configuration/#swap_size_mb-option) as +I would rather the machine run slower under peak load than crash. + +Finally, invest the time to set up a [WireGuard network](https://fly.io/docs/networking/private-networking/) that lets you VPN into your own private network of machines. + +## Multi-tenancy + +

+The code base supports only a single event. Running multiple events on a single machine is the next step before jumping into multiple machines. +

+ +There are a number of blog posts out there on how to do +[multi-tenancy](https://blog.arkency.com/comparison-of-approaches-to-multitenancy-in-rails-apps/) +with Rails. [Phusion Passenger](https://www.phusionpassenger.com/) provides a [different +approach](https://stackoverflow.com/questions/48669947/multitenancy-passenger-rails-multiple-apps-different-versions-same-domain). + +For this use case, the Phusion Passenger approach is the best match. The +database for a mid-size local event is about a megabyte in size (about the size +of a single camera image), and can be kept in SQLite. Passenger provides a +[`passenger_min_instances`](https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_min_instances) +`0` option that allow a reasonable number of past, present, and future events +to be hosted essentially without any overhead when not in use. This does mean +that you have to accept the cold start times of the first access, but that +appears to be two seconds or less on modern hardware, which is acceptable. + +The NGINX configuration file defines a set of environment variables for each tenant to control +the name of such things as the database, the log file, base URL, and pidfile. + +For web sockets (Action Cable), NGINX is [preferred over Apache +httpd](https://www.phusionpassenger.com/library/config/apache/action_cable_integration/). +The [documentation for Deploying multiple apps on a single server +(multitenancy)](https://www.phusionpassenger.com/library/deploy/nginx/) is +still listed as todo, but the following is what I have been able to figure out: + +- One action cable process is allocated per server (i.e., listen port). +- In order to share the action cable process, all apps on the same server will + need to share the same Redis URL and channel prefix. The [Rails + documentation](https://guides.rubyonrails.org/action_cable_overview.html#redis-adapter) + suggests that you use a different channel prefix for different applications + on the same server -- **IGNORE THAT**. +- Instead, use environment variables to stream from, and broadcast to, different + action cable channels. + +The end result is what outwardly appears to be a single Rails app, with a +single set of assets and a single cable. One additional rails instance +serves the index and provides a global administration interface. + +Recapping: a single server can host multiple events, each event is a separate instance of the same Rails application with a separate SQLite database placed on a volume. The server is always on (`auto_stop_machines = false`), but the Rails apps are spun up when needed and spin down when idle (using [`passenger_pool_idle_time`](https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_pool_idle_time)) + +## Multiple regions + +

+Distributing an app across multiple regions not only lets you service requests close +to your users, it enables horizontal scalability and isolation. +

+ +While most of my users are US based, my showcase app now has a second user in Australia, and now one in Warsaw, Poland. +Sending requests and replies half way around the world unnecessarily adds latency. + +As each machine is monolithic and self-contained, replicating service into a new region is +a matter of creating the new machine and routing requests to the correct machine. +This application use three separate techniques to route requests. + +The first technique is to [listen](https://github.com/rubys/showcase/blob/main/app/javascript/controllers/region_controller.js) for +[turbo:before-fetch-requests](https://turbo.hotwired.dev/reference/events#http-requests) events and +insert a [fly-prefer-region header](https://fly.io/docs/networking/dynamic-request-routing/#the-fly-prefer-region-request-header). +The region is extracted from a data attribute added to the [body](https://github.com/rubys/showcase/blob/aae08a6d57f92335b2cdbb94756e5416b7b50f83/app/views/layouts/application.html.erb#L16) tag. + +This all made possible by how [Turbo works](https://turbo.hotwired.dev/handbook/introduction#turbo-drive%3A-navigate-within-a-persistent-process): + +> This happens by intercepting all clicks on `` links to the same domain. When you click an eligible link, Turbo Drive prevents the browser from following it, changes the browser’s URL using the History API, requests the new page using fetch, and then renders the HTML response. + +The same source above also defines a `window.inject_region` function to inject the header in other places in the code that may issue fetch requests. + +The next (and final) two approaches are done in the NGINX configuration itself. This configuration is defined at startup on +each machine. Here is an example of the definition for `raleigh` on machines in regions other than `iad`: + +``` +## Raleigh +location /showcase/2024/raleigh/cable { + add_header Fly-Replay region=iad; + return 307; +} +location /showcase/2024/raleigh { + proxy_set_header X-Forwarded-Host $host; + proxy_pass http://iad.smooth.internal:3000/showcase/2024/raleigh; +} +``` + +This replays action cable (web socket) requests to the correct region, and +[reverse proxies](https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/) +all other requests to the correct machine using its [`.internal` address](https://fly.io/docs/networking/private-networking/#fly-io-internal-addresses). The latter is done because [fly-replay](https://fly.io/docs/networking/dynamic-request-routing/#limitations) +is limited to 1MB requests, and uploading audio files may exceed this limit. + +## Appliances + +

+For larger tasks, try +decomposing parts of your applications into event handlers, starting up Machines to handle requests when needed, and stopping them when the requests are done. +

+ +While above I've sung the praises of a Majestic Monolith, there are limits. Usage of this application +is largely split into two phases: a lengthy period of data entry, followed by a brief period of report generation. +This pattern then repeats a second time as scores are entered and then distributed. + +Report generation is done using [puppeteer](https://pptr.dev/) and [Google Chrome](https://www.google.com/chrome/). +While this app runs comfortably with 1MB of RAM, I've had Google Chrome crash machines that have 2MB. + +Generating a PDF from a web page (including authentication) is something that can be run independently of +all of the other services of your application. By making use of [`auto_stop_machines` and `auto_start_machines`](https://fly.io/docs/reference/configuration/#the-http_service-section), +a machine dedicated to this task can be spun up in seconds and perform this function. + +In my case, no changes to the application itself is needed, just a few lines of NGINX configuration: + +``` +## PDF generation +location ~ /showcase/.+\.pdf$ { + add_header Fly-Replay app=smooth-pdf; + return 307; +} +``` + +This was originally published as [Print on Demand](https://fly.io/blog/print-on-demand/), +and the code for that article can be found on [GitHub as fly-apps/pdf-appliance](https://github.com/fly-apps/pdf-appliance). + +This architectural pattern can be applied whenever there is a minority of requests that require an outsized amount +of resources. A second example that comes to mind: I've had requests for audio capture and transcription. +Setting up a machine that [runs Whisper with Fly GPUs](https://news.ycombinator.com/item?id=39417197) is +something I plan to explore. + +## Backups + +

+Running databases on a single volume attached to a single machine is a single +point of failure. It is imperative that you have recent backups in case of failure. +

+ +At this point, databases are small, and can be moved between servers with a simple configuration change. +For that reason, for the moment I've elected to have each machine contain a complete copy of all of the databases. + +To make that work, I +[install](https://github.com/rubys/showcase/blob/aae08a6d57f92335b2cdbb94756e5416b7b50f83/Dockerfile#L67), +[configure](https://github.com/rubys/showcase/blob/main/Dockerfile#L114-L137), and +[start](https://github.com/rubys/showcase/blob/aae08a6d57f92335b2cdbb94756e5416b7b50f83/bin/deploy#L26-L27) +[rsync](https://rsync.samba.org/). + +With Passenger, I set [`passenger_min_instances`](https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_max_pool_size) to zero, allowing +each application to completely shutdown after five minutes of inactivity. I define a +[`detached_process`](https://www.phusionpassenger.com/library/indepth/hooks.html#detached_process) hook to run +a [script](https://github.com/rubys/showcase/blob/main/bin/passenger-hook). + +As that hook will be run after each process exits, what it does first is query the number of running processes. If +there are none (not counting websockets), it begins the backup to each Fly.io machine +using `rsync --update` to only replace the newer files. After that completes, a webhook +is run on an offsite machine (a mac mini in my attic) to rsync all files there. In order for that to work, I run +[an ssh server](https://fly.io/docs/app-guides/opensshd/) on each Fly.io machine. + +And I don't stop there: + * My home machine runs the same passenger application configured to serve all of the events. + This can be used as a hot backup. I've never needed it, but it comforting to have. + * I also rsync all of the data to a [Hetzner](https://www.hetzner.com/) which + runs the same application. + * Finally, I have a cron job on my home machine that makes a daily backup of all of the databases. A new directory + is created per day, and [hard links](https://en.wikipedia.org/wiki/Hard_link) are used to link to the same file + in the all too common case where a database hasn't changed during the course of the day. As this is in a terabyte + drive, I haven't bothered to prune, so at this point, I have daily backups going back years. + +While this may seem like massive overkill, always having a local copy of all of the databases to use to reproduce problems +and test fixes, as well as having a complete instance that I can effectively use as a staging server alone makes all this +worth it. + +Finally, since each region contains a copy of all databases, I use [`auto_extend_size_threshold`](https://fly.io/docs/reference/configuration/#auto_extend_size_threshold) and +[`auto_extend_size_increment`](https://fly.io/docs/reference/configuration/#auto_extend_size_increment) to ensure that I never run out of space. + +## Logging + +

+It is important that logs are there when you need them. For this reason, logs need both monitoring and redundancy. +

+ +Equally as important as backups are logs. Over the course of the life of this application I've had two cases where +I've lost data due to errors on my part and was able to reconstruct that data by re-applying POST requests extracted +from logs. + +The biggest problem with logs is that you take them for granted. You see that they work and forget about them, and +there always is the possibility that they are not working when you need them. + +I've written about my approach in [Multiple Logs for Resiliency](https://fly.io/blog/redundant-logs/), but a summary here: + + * Since I have volumes already, I [instruct Rails to broadcast logs there](https://github.com/rubys/showcase/blob/aae08a6d57f92335b2cdbb94756e5416b7b50f83/config/environments/production.rb#L102-L110) in addition to stdout. + The API to do this changed in Rails 7.1, and the code I linked to does this both ways. This data isn't replicated to other volumes and + only contains Rails logs, as it really is only intended for when all else fails. + * I have a [logger](https://github.com/rubys/showcase/tree/main/fly/applications/logger) app that I run in two regions. + This writes log entries to volumes, and runs a small web server that I can use to browse and navigate the log entries. + This data too is backed up (forever) to my home server, and is only kept for seven days on the log server. + +Two major additions since I wrote up that blog entry: + * I have each machine [create a heartbeat log entry every 30 minutes](https://github.com/rubys/showcase/blob/main/bin/log-heartbeat.sh), and each log server runs a + [monitor](https://github.com/rubys/showcase/blob/main/fly/applications/logger/monitor.ts) once an hour to + ensure that every machine has checked in. If a machine doesn't check in, a [Sentry](https://sentry.io/) + alert is generated. + * I have my log tracker query Sentry whenever I visit the web page to see if there are new events. This + serves as an additional reminder to go check for errors. + +I strongly encourage people to look into [Sentry](https://community.fly.io/t/integrated-sentry-error-tracking/15278). + +## Applying updates + +

+While Fly.io will start machines fast, you have to do your part. +If your app doesn't start fast, start something else that starts quickly first +so your users see something while they are waiting. +

+ +If you go to https://smooth.fly.dev/ the server nearest to you will need to spin up the index application. And if, from there, you click on demo, a second application will be started. While you might get lucky and somebody else (or a bot) may have already started these for you, the normal case is that you will have to wait. + +Since all the services that are needed to support this application are on one machine, `fly deploy` can't simply start a new machine, wait for it to start, direct traffic to it, and only then start dismantling the previous machine. Instead, each machine needs to be taken offline, have the rootfs replaced with a new image, and all of its services restarted. Once that is complete, migrations need to be run on that machine; and in my case I may have several SQLite databases that need to be upgraded. I also have each machine that comes up rsync with the primary region. And finally, I generate an operational NGINX configuration and run [nginx reload](https://docs.nginx.com/nginx/admin-guide/basic-functionality/runtime-control/). + +While this overall process may take some time, NGINX starts fast so I start it first with a [configuration](https://github.com/rubys/showcase/blob/main/config/nginx.startup) that directs everything to a [Installing updates..](http://smooth.fly.dev/showcase/503.html) page that auto refreshes every 15 seconds which normally is ample time for all this to occur. Worst case, this page simply refreshes again until the server is ready. + +Even with this in place, it still is worthwhile to make the application boot fast. Measuring the times of the various components of startup, in my case Rails startup time dominates, and it turns out that effectively `bin/rails db:prepare` starts Rails once for each database. +I found that I can avoid that by [getting a list of migrations](https://github.com/rubys/showcase/blob/cb3a05fd48c22a140effcf03ccfa9bd038a0df30/config/tenant/nginx-config.rb#L186), +and then [comparing that list to the list of migrations already deployed](https://github.com/rubys/showcase/blob/cb3a05fd48c22a140effcf03ccfa9bd038a0df30/config/tenant/nginx-config.rb#L207-L215). +I also moved preparation of my demo database to [build time](https://github.com/rubys/showcase/blob/cb3a05fd48c22a140effcf03ccfa9bd038a0df30/Dockerfile#L139-L140). In most cases, this means that my users +see no down time at all, just an approximately two second delay on their next request as Rails restarts. + +## Administration + +

+While Fly.io provides a [Machines API](https://fly.io/docs/machines/working-with-machines/), +you can go a long way with old-fashioned scripting using the [flyctl](https://fly.io/docs/flyctl/) +command. +

+ +Now lets look at what happens when I create a new event, in a new region. I (a human) get an email, I bring up my admin UI (on an instance of the index app I mentioned above). I fill out a form for the new event. I go to a separate page and add a new region. I then go to a third form where I can review and apply the changes. This involves regenerating my map, and I need to update the NGINX configuration in every existing region. There are shortcuts I can utilize which will update files on each machine and reload the NGINX configuration, but I went that way I would need to ensure that the end result exactly matches the reproducible result that a subsequent `fly deploy` would produce. I don't need that headache, so I'm actually using `fly deploy` which can add tens of seconds to the actual deploy time. + +I started out running the various commands manually, but that grew old fast. So eventually +I invested in creating an admin UI. + +Here's what the front page of the admin UI looks like: + +Showcase Administration + +From there, I can add, remove, and reconfigure my events, and apply changes. I can also visit +this app on three different servers, and see the logs from each. Finally, I have links to handy +tools. The Sentry link that is current gray turns red when there are new events. + +I'm actually very happy with the result. The fact that I could get an email and add an event and have everything up and running on a server half a world away in less than five minutes is frankly amazing to me. +And to be honest, most of that is waiting on the human (me), followed by the amount of time it takes to build and deploy a +new image in nine regions. + +And in most cases, I can now make and deploy changes from my smartphone rather than requiring +my laptop. + +For those who are curious, here's the [deployment script](https://github.com/rubys/showcase/blob/main/bin/apply-changes.rb). + +## Costs + +

+This is a decent sized personal project, deployed across three continents. Even with a +significant amount of over-provisioning, the costs per month are around $60, before taking +into account plans and allowances. +

+ +I'm currently running this app in nine regions. I have two machine running a log server. +I have two machines that will run puppeteer and Chrome on demand. + +[Fly.io pricing](https://fly.io/docs/about/pricing/) includes a number of plans which include a number of allowances. To +keep things simple, I'm going to ignore all of that and just look at the total costs. +From there, add your plan and subtract your allowances to see what this would cost for you. + +And be aware that things may change, the below numbers are current as of early March, 2024. + +Let's start with compute. 9 shared-cpu-1x machines with 1GB each at $5.70 per month total $51.30. +Two shared-cpu-1x machines with 512MB each at $3.19 per month total $6.38. +The print on demand machines are on bigger machines, but are only run when requested. +I've configured them to stay up for 15 minutes when summoned. Despite all of this the +costs for a typical month are around $0.03. So we are up to $57.71. + +Next up volumes. I have 9 machines with 3GB disks, and 2 machines that only need 1G each. +29GB at $0.15 per month is $1.35. + +I have a dedicated IPv4 address in order to use ssh. This adds $2.00 per month. + +Outbound transfer is well under the free allowance, but for completeness I'm seeing about 16GB in North America and Europe, +which at $0.02 per GB amounts to $0.32. And about 0.1GB in Asia Pacific, which at $0.04 per GB adds less than a cent, +but to be generous, let's accept a total of $0.33. + +Adding up everything, this would be 57.71 + 1.35 + 2.00 + 0.33 for a total of $61.39 per month. + +Again, this is before factoring in plans and allowances, and reflects the current pricing in March of 2023. + +For my needs to date, I'm clearly over-provisioning, but it is nice to have a picture of what +a fully fleshed out system would cost per month to operate. + +## Summary + +

+Shared Nothing architectures are a great way to scale up applications where the data involved is easily partitionable. +

+ +Key points (none of these should be a surprise, they were highlighted +in the text above): + +* Run as many services as you like on a single machine; this both increases throughput and reliability. +* You can even run multiple instances of the same app on a machine. +* Distributing your apps across multiple regions lets you service requests close to your users. +* For larger tasks, start up machines on demand. +* Every backup plan should have a backup plan. +* Not only should you have logs, logs should be monitored and have redundancy. +* Make sure that you start something quickly -- even if it is only a "please wait" placeholder -- to make sure that your app is always servicing requests. +* Create an administration UI and write scripts you can use to change your configuration. + +And finally, this app currently supports 77 events in 36 cities with 9 servers for less than you probably pay for a dinner out a month or a smart phone plan. + +## Background + +

+Feel free to skip or skim this section. It describes how the application came to be, what it does, and a high level description of its usage needs. +

+ +Background information on how this application came to be, and got me involved with Fly.io can be found +at [Unretiring](http://intertwingly.net/blog/2022/08/13/Unretiring) on my personal blog. + +Here's an excerpt from that post: + +> I was at a local ballroom dance competition and found myself listed twice in one heat - a scheduling mishap where I was expected to be on the floor twice with two different dance partners. +> +> After the event, Carolyn and I discovered that the organizers used a spreadsheet to collect entries and schedule heats. A very manual and labor prone process, made more difficult by the need to react to last minute cancellations due to the ongoing pandemic. +> +> Carolyn told the organizers that I could do better, and volunteered me to write a program. I thought it would be fun so I agreed. + +This application is [on GitHub](https://github.com/rubys/showcase#readme), and while each individual event requires authentication for access, a full function demo can be accessed at [https://smooth.fly.dev/](ht +tp://smooth.fly.dev/). It was originally deployed a single Mac mini, and now is running on a small fleet of machines in the Fly.io network. + +--- + +Couples ballroom dance competitions are called a "Showcase". +The application supports: + +- managing all the contestants +- storing and playing dance music for solos +- invoicing students and studios +- managing the schedule +- printing heat sheets, back numbers, and labels +- recording the judges' scores +- publishing results + +As you might expect, a Showcase competition happens live and in one location. There are very few users who need access to the system and they often are all physically in the same location. +Here we'll cover how a Ruby on Rails application using a SQLite database can be deployed to the nearest Fly.io region for the event and how elegantly it solved this problem, including the ability to easily deploy new servers from the application. + +--- + +The core Showcase application is implemented on top of Ruby on Rails, augmented by two separate services implemented in JavaScript an run using [Bun](https://bun.sh/). While the details may vary, many of the core principles described below apply to other applications, +even applications using other frameworks. In fact, many of the pieces of advice provided below apply to apps you host yourself, or host on +other cloud providers. + +But first, some useful approximations that characterize this app, many of which will guide the choice of architecture: + +* Each event is a separate instance of the Rails application, with a its own separate database. +* A typical database is one megabyte. Not a typo: one megabyte. Essentially, the app is a spreadsheet with a [Turbo](https://turbo.hotwired.dev/handbook/introduction) web front end. +* All the services needed to support an event are run on a single VM. +* Persistence consists of the database, storage (used for uploaded audio files), and logs. +* Typical response time for a request is 100ms or less. +* Peak transaction per second per database is approximately 1. Again, not a typo -- think spreadsheet. +* Peak number of simultaneous users is often one, but even when there are multiple they tend to be all in the same geographic region, often in the same room (example: multiple judges entering scores). +* Transactions are generally a blend of reads and writes with neither dominating. + +This lends itself well to horizontal scaling, where every event effectively has a dedicated VM. Again, this is a useful approximation - as not every event is "active", multiple events can share a VM. As needs change, databases can be moved to different VMs. + +## Related reading + +- [Getting Started with N‑Tier Architecture](/docs/blueprints/n-tier-architecture/) A contrasting architecture pattern where state is shared across the data tier — useful for understanding when shared‑nothing *isn’t* the best fit. +- [Session Affinity (a.k.a. Sticky Sessions)](/docs/blueprints/sticky-sessions/) Focused on routing and state‑isolation patterns that often come up when you adopt a shared‑nothing approach and need to keep state local to a machine. +- [Networking](/docs/networking/) The comprehensive networking overview (private networks, Anycast, WireGuard tunnels, routing) — helpful when building globally distributed isolated nodes in a shared‐nothing architecture. \ No newline at end of file diff --git a/blueprints/staging-prod-isolation.html.md b/blueprints/staging-prod-isolation.html.md new file mode 100644 index 0000000000..982d2658c3 --- /dev/null +++ b/blueprints/staging-prod-isolation.html.md @@ -0,0 +1,54 @@ +--- +title: Staging and production isolation +layout: docs +nav: guides +redirect_from: /docs/going-to-production/the-basics/production-staging-isolation/ +--- + +
+ Illustration by Annie Ruygt of a gateway and buildings +
+ +## Overview + +You want to isolate your production environment from your test or staging environments by limiting production access to the smallest possible group. You can get dependable isolation by using multiple organizations. Organization access is invitation-only and orgs are isolated from one another at a low level by our private networking architecture: every organization has its own [private network](/docs/networking/private-networking/) by default. + +All Fly.io accounts start with a "personal" organization. You can create as many organizations as you need, not just for different environments, but also for different projects or clients. + +Before you create a new organization, here's what you need to know: + +- You can consolidate billing for multiple organizations under a a single parent organization, or you can create regular organizations with separate payment methods and billing. Learn more about [unified billing](/docs/about/billing/#unified-billing). +- You invite or remove members in each organization separately. +- An organization can have multiple apps and those apps can communicate with each other securely over the org's private network. + +Adjust the pattern to fit your needs. For example: + +- personal organization: the org you started with, and the one you keep for personal projects +- staging organization: `-staging` used for development and testing +- production organization: `-production` used for your production app + +## Work with multiple organizations + +It's best if you use one Fly.io account to manage all your organizations, so you can access them without logging in and out. App names are unique across the Fly.io platform, so your staging and production apps will have different names and their own `fly.toml` files for configuration. + +### Manage organizations in your dashboard + +To create a new org from the [dashboard]((https://fly.io/dashboard/)), select **Create new organization** from the **Organization** dropdown. + +To view or send invites to members, use the **Organization** dropdown to choose an org, then go to **Team**. + +### Manage organizations with flyctl + +To create new organizations and invite or remove members, use [`fly orgs` flyctl commands](/docs/flyctl/orgs/). + +When you have more than one org, flyctl prompts you to choose an organization when required. You can run commands on a specific app in any org using the `--app` option if you aren't in the app's project source directory in your terminal. + +## Other kinds of isolation and access control + +- When you only need app isolation within an org, you can use [custom private networks](/docs/networking/custom-private-networks/) to isolate apps from one another by creating an app with `fly apps create` and the `--network` option. +- When you want user or 3rd-party access control, you can use [deploy](https://community.fly.io/t/deploy-tokens/11895) and [org-scoped tokens](https://community.fly.io/t/org-scoped-tokens/13194) to limit access to apps or orgs. + +## Related reading + +- [Going to production checklist](/docs/apps/going-to-production/) A broad checklist for production‑readiness that includes isolating environments and access control. +- [Staging environments with GitHub actions](/docs/django/advanced-guides/staging-environments-with-github-actions/) A practical guide focused on using GitHub Actions to deploy dedicated staging environments. Complements the isolation pattern nicely. \ No newline at end of file diff --git a/blueprints/sticky-sessions.html.md b/blueprints/sticky-sessions.html.md new file mode 100644 index 0000000000..770a8bbfe9 --- /dev/null +++ b/blueprints/sticky-sessions.html.md @@ -0,0 +1,156 @@ +--- +title: Session Affinity (a.k.a. Sticky Sessions) +layout: docs +nav: guides +author: rubys +date: 2024-07-17 +--- + +
+ Illustration by Annie Ruygt of a machine sticking envelopes together +
+ +## Overview + +In a number of scenarios, it is important to ensure that certain requests are routed to a specific Machine. This frequently is expressed in the form of wanting an entire user's session to be processed by the same Machine. + +There are two approaches to addressing this with Fly.io: + * [fly-force-instance-id](#fly-force-instance-id) - Client-side request header + * [fly-replay](#fly-replay) - Server-side response header + +## Fly-Force-Instance-Id + +This approach requires you to have control over the client, typically a browser, but allows for immediate routing without an additional hop. The example below uses Rails' Hotwire [Turbo](https://turbo.hotwired.dev/) with [Stimulus](https://stimulus.hotwired.dev/) to send the required header. + +For this to work, the client needs some way of knowing what Machine to route requests to. This can be accomplished by adding attributes to the `` tag in HTML responses. With Rails, those tags would be found in +`app/views/layouts/application.html.erb`. + +An example `` tag: + +```erb + +``` + +These attributes identify the Machine instance to direct future requests to, and the name of the Stimulus controller to be used to make this happen. + +Place the following into `app/javascript/controllers/sticky-session.js`: + +```js +import { Controller } from "@hotwired/stimulus" + +// Connects to data-controller="sticky-session" +export default class extends Controller { + connect() { + document.documentElement.addEventListener( + 'turbo:before-fetch-request', + this.beforeFetchRequest + ) + } + + disconnect() { + document.documentElement.removeEventListener( + 'turbo:before-fetch-request', + this.beforeFetchRequest + ) + } + + beforeFetchRequest = event => { + event.detail.fetchOptions.headers['fly-force-instance-id'] = + this.element.dataset.instance; + } +} +``` + +This code will subscribe and unsubscribe from +[turbo:before-fetch-request](https://turbo.hotwired.dev/reference/events#turbo%3Abefore-fetch-request) events and insert a `fly-force-instance-id` header into requests. + +## Fly-Replay + +This approach is implemented entirely on the server. Your application examines each request, determines which Machine should handle it, and returns a `fly-replay` response header to route the request accordingly. Initial requests will require an additional "hop" to route, and [requests are limited to payloads of 1 megabyte](https://fly.io/docs/networking/dynamic-request-routing/#limitations). + +The example below creates [Express Middleware](https://expressjs.com/en/guide/using-middleware.html) that routes requests based on a session cookie: + +```js +// Import required modules +import express from "express"; +import cookieParser from "cookie-parser"; + +// Create an Express app +const app = express(); + +// Middleware to parse cookies +app.use(cookieParser()); + +app.use((request, response, next) => { + // This assumes your application has already established a session using a session_id cookie via your session middleware. + const sessionId = request.cookies["session_id"]; + + if (!sessionId) { + // No session - let any machine handle it + next(); + return; + } + + // Determine which machine should handle this session + // This could be based on: + // - Database lookup (session -> machine mapping) + // - Consistent hashing of session ID + // - Regional routing logic + // - Load balancing algorithm + // You'll need to implement determineTargetMachine() based on your routing strategy + const targetMachineId = determineTargetMachine(sessionId); + + if (targetMachineId === process.env.FLY_MACHINE_ID) { + // This is the correct machine - handle the request + next(); + } else { + // Route to the correct machine + response.set('Fly-Replay', `instance=${targetMachineId}`); + response.status(307); + response.send(); + } +}); + +// Start the Express server +const port = 3000; +app.listen(port, () => { + console.log(`Server is running on port ${port}`); +}); +``` + +### Optimizing with Replay Caching + +For production workloads, having your application replay every request can create unnecessary load and latency. You can configure Fly Proxy to cache replay decisions, so only the first request in a session needs to consult your application. + +Add replay cache rules to your `fly.toml`: + +```toml +[http_service] + internal_port = 8080 + force_https = true + + [[http_service.http_options.replay_cache]] + path_prefix = "/" + ttl_seconds = 300 + type = "cookie" + name = "session_id" +``` + +With this configuration: +1. The first request with a given `session_id` hits your app +2. Your app returns a `fly-replay` response +3. Fly Proxy caches this decision for that specific `session_id` value +4. For the next 5 minutes, requests with the same `session_id` are automatically routed without consulting your app + +Caching is an optimization, not a guarantee, so your origin app still needs to handle replaying requests that aren't routed by a cached replay. + +Replays can be cached on any existing cookie or request header. For sticky sessions with an API, for example, you might choose to cache based on the `Authorization` header. + +For complete details on replay caching configuration, see [Session-based Replay Caching](/docs/networking/dynamic-request-routing/#session-based-replay-caching). + +## Related reading + +- [Dynamic Request Routing with `fly‑replay`](/docs/networking/dynamic-request-routing/) Explains how `fly‑replay` lets you route requests to a specific machine or region—core technique for sticky sessions. +- [Load Balancing](/docs/reference/load-balancing/) Details how the Fly Proxy routes traffic among Machines, which underpins how session‑affinity decisions get made. +- [Networking](/docs/networking/) A broad overview of how Fly handles public/private networking, Anycast IPs, WireGuard mesh, domain routing, and more. \ No newline at end of file diff --git a/blueprints/supercronic.html.md b/blueprints/supercronic.html.md new file mode 100644 index 0000000000..bd33e9258a --- /dev/null +++ b/blueprints/supercronic.html.md @@ -0,0 +1,109 @@ +--- +title: Crontab with Supercronic +layout: docs +nav: guides +author: brad +categories: + - cron +date: 2022-11-19 +redirect_from: + - /docs/app-guides/superchronic/ + - /docs/app-guides/supercronic/ +--- + +
+ Illustration by Annie Ruygt of Framkie the balloon wearing a sash and a prize ribbon in a gymnasium hall while a bird looks with admiration +
+ +## Overview + +`crontab` is a little too opinionated for containers—it's a great little tool, but when run inside of containers it doesn't grab `ENV` vars how we'd like it to. Fortunately there's a Go binary called `supercronic` that's a drop-in replacement for containers. + +The good news is that it's pretty easy to get it running on Fly with a little bit of copy and pasting. Let's get to it. + +## Create a crontab file + +In the root of your project, add a `crontab` file. + +``` +touch ./crontab +``` + +If you need to run a job every 5 minutes, your `crontab` file would something like this: + +``` +*/5 * * * * echo "hello world!" +``` + +Check out [cron.help](https://cron.help+external) if you need a quick crontab syntax reference. + +## Install `supercronic` in the container + +The [latest releases for supercronic](https://github.com/aptible/supercronic/releases+external) are on [GitHub](https://github.com/aptible/supercronic+external), where they include copy pasta 🍝instructions for getting it in your Dockerfile. As of January 2024, the current version of `supercronic` is v0.2.29. You'll want to check the [releases page](https://github.com/aptible/supercronic/releases+external) for the latest version, but here's what it looks like now: + +``` +# Latest releases available at https://github.com/aptible/supercronic/releases +ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.29/supercronic-linux-amd64 \ + SUPERCRONIC=supercronic-linux-amd64 \ + SUPERCRONIC_SHA1SUM=cd48d45c4b10f3f0bfdd3a57d054cd05ac96812b + +RUN curl -fsSLO "$SUPERCRONIC_URL" \ + && echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \ + && chmod +x "$SUPERCRONIC" \ + && mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \ + && ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic + +# You might need to change this depending on where your crontab is located +COPY crontab crontab +``` + +After you put that in your Dockerfile, we're going to configure Fly to run only one instance of `supercronic`. Why? Because if you have a cronjob that does something like deliver emails to customers, you only want them to get that once. The last thing you want is them getting as many emails from your services that matches the number of instances you're running. + +## Setup a web & cron process + +At the bottom of the `fly.toml` file, add the following: + +``` +[processes] + # The command below is used to launch a Rails server; be sure to + # replace with the command you're using to launch your server. + web = "bin/rails fly:server" + cron = "supercronic /app/crontab" +``` + +Then we have to tell Fly that your `web` process matches up with a service by having this under the `[[services]].` + +``` +# Make sure there's only one `processes` entry under `[[services]]` +[[services]] + processes = ["web"] +``` + +This change tells Fly that your `web` processes that you defined under `[processes]` will be exposed to the public internet. Your `cron` process won't be exposed to the public Internet because it doesn't need to! + +## Deploy & scale the new processes + +Now that we've added `supercronic` to our `Dockerfile`, put the `crontab` at the root of our project folder, and reconfigured the `fly.toml` file, we're ready to deploy to production. + +``` +$ fly deploy +``` + +Then we'll need to scale the processes so that we only run one virtual machine container with the `cron` process. Be sure to change `web` to whatever number you had before. + +``` +$ fly scale count cron=1 web= +``` + +That's it! If all went well you now have cron running in a `cron` process in a Fly virtual machine. When you `fly deploy` it will get the latest code changes and reboot the virtual machines. + +## Resources + +- [https://github.com/fly-apps/supercronic](https://github.com/fly-apps/supercronic+external) +- [https://cron.help](https://cron.help+external) + +## Related Reading + +- [Task scheduling guide with Cron Manager and friends](/docs/blueprints/task-scheduling/) A broader look at scheduling jobs on Fly.io — covers Cron Manager, Supercronic, scheduled Machines, and more. +- [Run multiple process groups in an app](/docs/launch/processes/) Explains how to define processes in `fly.toml`, run multiple programs (web, cron, workers), scale by group. +- [App Configuration (`fly.toml`)](/docs/reference/configuration/) In‑depth look at the config file that orchestrates your services, processes, health checks, etc. \ No newline at end of file diff --git a/blueprints/task-scheduling.html.md b/blueprints/task-scheduling.html.md new file mode 100644 index 0000000000..b5755ddd44 --- /dev/null +++ b/blueprints/task-scheduling.html.md @@ -0,0 +1,149 @@ +--- +title: Task scheduling guide with Cron Manager and friends +layout: docs +nav: guides +author: kcmartin +categories: + - cron +date: 2025-07-18 +--- + +
+ Illustration by Annie Ruygt of Frankie the hot air balloon scheduling some tasks on his calendar +
+ +## Overview + +Running on a schedule: it's not just for trains. If your app needs to rebuild a cache every night, prune old data, or email a weekly newsletter to your users, you’re probably looking for a way to run a cron job. Fly.io gives you a few ways to get that done. + +We’ll walk through options for task scheduling, starting with the most robust and ending with the “it works, okay?” tier. + +## Cron Manager + +[Cron Manager](https://github.com/fly-apps/cron-manager) is our “batteries-included” solution for running scheduled jobs. It’s a small Fly app that spins up temporary Machines—one per job—and tears them down afterward. Think of it as a very polite butler that rings a bell and quietly disappears before you remember you don’t have a butler. + +### How It Works + +You deploy Cron Manager as its own Fly app. It stays online, watching a `schedules.json` file. When a job’s scheduled time hits, it boots a one-off Machine with whatever image, region, and resources you specify. That Machine runs the command, exits, and disappears like a good container should. + +Jobs run in complete isolation—no shared environment, no weird state carried over from the last job. And because they’re just Fly Machines, you get all the perks: region placement, resource control, image selection, and clean logs. + +### Why Use This + +- **Isolation**: Each job gets its own Machine. Nothing leaks. +- **Central config**: `schedules.json` lives in Git. Changes are versioned, auditable, and deployable. +- **Easy updates**: Want to change a command or schedule? Update the file and redeploy. +- **Job logs**: Each Machine has its own logs and lifecycle. Debugging isn’t a treasure hunt. + +### Setup + +1. Clone [cron-manager](https://github.com/fly-apps/cron-manager) +1. Create a new Fly app and deploy it with `fly deploy` +1. Set your `FLY_API_TOKEN` as a secret for the app +1. Define your job schedules in `schedules.json` + +Here’s a sample job: + +```json +{ + "name": "daily-cache-rebuild", + "app_name": "my-app", + "schedule": "0 3 * * *", + "region": "lhr", + "command": ["bin/rebuild-cache"], + "command_timeout": 300, + "enabled": true, + "config": { + "image": "my-app-image", + "guest": { "cpu_kind": "shared", "cpus": 1, "memory_mb": 256 }, + "restart": { "policy": "no" } + } +} +``` + +### Managing Jobs + +Once deployed, you can: + +- List jobs: `cm schedules list` +- Check history: `cm jobs list ` +- Peek at details: `cm jobs show ` +- Trigger one manually: `cm jobs trigger ` + +SSH into the Cron Manager VM if you need to do some spelunking. + +--- + +## Supercronic + +Sometimes you don’t need a full orchestration layer—you just want a crontab to run inside your app container. Supercronic does that. It’s a container-friendly cron runner that handles environment variables correctly. + +### How It Works + +[Supercronic](/docs/blueprints/supercronic/) runs as a background process inside your app. You add a `crontab` file, install Supercronic in your Dockerfile, and define a `cron` process in `fly.toml`. Done. + +Well, almost done—remember to scale it correctly. + +### Setup + +1. Add a crontab file (`/app/crontab`) using standard syntax +1. Install Supercronic in your Dockerfile +1. In `fly.toml`: + +``` +[processes] +app = "bin/server" +cron = "supercronic /app/crontab" +``` + +1. Scale it: + +```bash +fly scale count app=2 cron=1 +``` + +Only run one copy of the cron process. Unless your goal is to send four copies of the same reminder email. (In which case, carry on.) + +--- + +## Scheduled Machines + +Fly Machines support basic scheduling out of the box: you can tell a Machine to start hourly, daily, weekly, or monthly. The API is simple. The downside? You don’t get fine-grained control—just interval buckets. + +This is fine for non-critical cleanup tasks or anything where “roughly once a day” is good enough. + +--- + +## In-App Scheduling + +If you’re already using a scheduler like `apscheduler` in a Python app—or you’ve duct-taped one together in Node—you can use it to call `fly machine run --rm` and spin up ephemeral Machines manually. + +This gives you full control and is technically elegant, though it does require some careful footwork. It’s the “build your own trigger system” option, best for when you’re already halfway there. + +--- + +## Additional Options + +In addition to the options above, you can use an external scheduling service, like [GitHub Actions schedule](https://docs.github.com/en/actions/reference/events-that-trigger-workflows#schedule), or [easycron](https://www.easycron.com/), an online cron service which can send out API requests to wake machines up. + +--- + +## Summary + +| Option | Use When… | Good To Know | +| --- | --- | --- | +| **Cron Manager** | You want isolated, auditable job runs | Requires a separate app | +| **Supercronic** | You want quick cron jobs inside your container | Keep the process count to one | +| **Scheduled Machines** | You want simple, low-frequency jobs (~1x/day) | Not for precise timing | +| **In-App Scheduler** | You want full control and don't mind plumbing | Write your own orchestration | + +If you're not sure where to start, Cron Manager is the most production-hardened option. But Supercronic gets you 80% of the way with almost no setup, and sometimes that’s all you need. + +--- + +## Related Reading + +- [Cron Manager](https://github.com/fly-apps/cron-manager) +- [Crontab with Supercronic](/docs/blueprints/supercronic/) +- [Deferring long-running tasks to a work queue](/docs/blueprints/work-queues/) + diff --git a/blueprints/using-base-images-for-faster-deployments.html.md b/blueprints/using-base-images-for-faster-deployments.html.md new file mode 100644 index 0000000000..08b3e6096d --- /dev/null +++ b/blueprints/using-base-images-for-faster-deployments.html.md @@ -0,0 +1,272 @@ +--- +title: Using base images for faster deployments +layout: docs +nav: guides +--- + +
+ Illustration by Annie Ruygt of the Docker whale carrying some containers +
+ +## Overview + +This guide explains how to use base images for faster deployments. A base image is any image that is intended to be used as the `FROM` line in a Dockerfile. Each app that is deployed on Fly.io can add an image to the Fly registry. + +Every app in the same organization can access the image for any other app and use it as the `FROM` line in their Dockerfile. This means that it is very easy to make a base image by making a second app to use as the base image. + +## Why use a base image? + +Most developers have a specific reason for using a base image in their project. Usually it's one of the following: + +* The base image can be built with all of their app's dependencies. Rebuilding the app on top of this base image means the deploy process is only running the app-specific build processes, not the entire dependency tree. +* The same app is deployed for a number of different end users. Each end user has some specific files or configurations to build into the image. A base image can get *most of the way* to the final configuration, and their app can do the final steps. + + +## How to make a base image + +This guide walks through building and deploying an example application called go-fly-a-site, which runs a simple Go web server. After deploying the app, you'll create a base image from its non-app-specific parts. Finally, you'll update the app to use that base image. + +### Making the app + +Our program will be very simple. It will respond to HTTP requests on port 8080 and respond with "Hello World!". + +```GO +package main + +import ( + "fmt" + "net/http" +) + +func main() { + http.HandleFunc("/", HelloServer) + http.ListenAndServe(":8080", nil) +} + +func HelloServer(w http.ResponseWriter, r *http.Request) { + fmt.Fprint(w, "Hello, world!\r\n") +} + +``` + +Create a Dockerfile based on Debian 12. Install curl and go, compile the Go code, and configure the image to run the compiled binary. + +```Dockerfile +FROM debian:12 + +# Install Go +RUN apt-get update; apt-get upgrade; apt-get -y install golang + +# Copy the code for our server to /opt/main.go +COPY main.go /opt/main.go + +# Build /opt/server from the code in /opt/main.go +RUN go build -o /opt/server /opt/main.go + +# Run our code from main.go as /opt/server +CMD [ "/opt/server" ] +``` + +With `main.go` and the `Dockerfile` in your project directory, deploy the application: + +```bash +$ fly launch --name go-fly-a-site --vm-size shared-cpu-1x --no-deploy +.... +Your app is ready! Deploy with `flyctl deploy` +``` + +Now we have an additional file, `fly.toml` and have created the app `go-fly-a-site`. + +Deploy the app: + +```bash +$ fly deploy +==> Verifying app config +Validating /Users/fly/Code/go-fly-a-site/fly.toml +✓ Configuration is valid +--> Verified app config +==> Building image +==> Building image with Depot +--> build: +... + +Visit your newly deployed app at https://go-fly-a-site.fly.dev/ + +``` + +We can see it's running: + +```bash +$ curl https://go-fly-a-site.fly.dev/ +Hello, world! +``` + +Now our app is built and running. This is great, but each time we `fly deploy` we might be reinstalling Go through apt. We probably want to do that much less often than we want to update our `main.go`, so we can make a base image that installs go for us. + +### Make a base image + +Let's make a new file called `base.Dockerfile`. For this file, we will keep the Go install. We will replace the previous `CMD` statement with `[ "sleep", "inf" ]` so that we can easily SSH into this image to inspect it if needed in the future. + + +```Dockerfile +FROM debian:12 + +# Install Go +RUN apt-get update; apt-get upgrade; apt-get -y install golang + +CMD [ "sleep", "inf" ] +``` +Copy `fly.toml` to `fly-base.toml` and make the following modification: + +* Append -base to the app name. +* Explicitly include dockerfile in the `[build]` section. +* Remove the `[http_service]` section. + +The resulting file looks like this: + +```toml +app = 'go-fly-a-site-base' +primary_region = 'lax' + +[build] + dockerfile = "base.Dockerfile" + +[[vm]] + size = 'shared-cpu-1x' +``` + +Create the app using the following options: + +* `--ha=false` ensures the app uses a single Machine. +* `--config base.fly.toml` specifies the correct config file. + +When prompted about the existing `fly.toml` file, select yes to copy its configuration. You can skip tweaking the settings. + +```bash +$ fly launch --ha=false --config base.fly.toml +An existing fly.toml file was found for app go-fly-a-site-base +? Would you like to copy its configuration to the new app? Yes +Using build strategies '[the "base.Dockerfile" dockerfile]'. Remove [build] from fly.toml to force a rescan +Creating app in /Users/fly/Code/go-fly-a-site +We're about to launch your app on Fly.io. Here's what you're getting: + +... +Region: Los Angeles, California (US) (from your fly.toml) +App Machines: shared-cpu-1x, 256MB RAM (from your fly.toml) +... + +? Do you want to tweak these settings before proceeding? (y/N) +Created app 'go-fly-a-site-base' in organization 'personal' +Admin URL: https://fly.io/apps/go-fly-a-site-base +Hostname: go-fly-a-site-base.fly.dev +Wrote config file base.fly.toml +... +--> build: +--> Building image done +image: registry.fly.io/go-fly-a-site-base:deployment-01JTKF7JXJ1ZGMA76GCJ5WGZQZ +image size: 233 MB +``` + +Some things to note: + +* We have the base image name we can use, it's the URL provided in `image:`. +* When we go to the admin URL, we can click **Registry** and see the same image URL. +* The **Registry** will have the list of images that have been created. +* Any app in your organization can use the image URL for a `FROM` statement in a Dockerfile. +* Apps in any other organization will get an error if they try to use this image. + + +### Updating the app to use the base image + +After creating the base image, update the Dockerfile for go-fly-a-site: + + +```Dockerfile +FROM registry.fly.io/go-fly-a-site-base:deployment-01JTKF7JXJ1ZGMA76GCJ5WGZQZ + +# Copy the code for our server to /opt/main.go +COPY main.go /opt/main.go + +# Build /opt/server from the code in /opt/main.go +RUN go build -o /opt/server /opt/main.go + +# Run our code from main.go as /opt/server +CMD [ "/opt/server" ] +``` + +With the updated Dockerfile in place, deploy the app with `fly deploy`. + +```bash +$ fly deploy +==> Verifying app config +Validating /Users/fly/Code/go-fly-a-site/fly.toml +✓ Configuration is valid +--> Verified app config +==> Building image +==> Building image with Depot +--> build: +[+] Building 20.7s (8/8) FINISHED + ... + => [internal] load metadata for registry.fly.io go-fly-a-site-base:deployment-01JTKF7JXJ1ZGMA76GCJ5WGZQZ + ... + ``` + + We can see that it used the new base image, and the application is running: + +```bash +$ curl https://go-fly-a-site.fly.dev/ +Hello, world! +``` + +We can shutdown the one Machine for the base image: + +``` +$ fly machine list --config base.fly.toml +1 machines have been retrieved from app go-fly-a-site-base. +View them in the UI here + +go-fly-a-site-base +ID NAME ... +d890175f6940e8 wandering-wave-6179 ... +```` + +Grab the Machine ID and then run `fly machine stop` with the app and Machine id: + +```bash +$ fly machine stop d890175f6940e8 --config base.fly.toml +Sending kill signal to machine d890175f6940e8... +d890175f6940e8 has been successfully stopped +``` + +You’ll only be charged for the root file system storage of the image that's being stored. + +Now we have a base image for our project! We can modify and deploy our `main.go` code without having to reinstall Go. If we ever need to add additional dependencies to the base image, we just modify the `base.Dockerfile`, deploy it again, and get the new `image:` from the deploy command or in the **Registry** section of the app's dashboard. + +## Command Reference + +Get Machines for the base image: `fly machine list --config base.fly.toml` + +Start Machine for the base image: `fly machine start --config base.fly.toml` + +Stop Machine for the base image: `fly machine stop --config base.fly.toml` + +Get SSH access to the base image: `fly ssh console --config base.fly.toml` (Make sure the Machine is started) + +Rebuild the base image: `fly deploy --config base.fly.toml` (Update `base.Dockerfile` with your changes first, and remember to stop the Machine when you're done) + +## Additional Notes + +If you have a base image that many different apps will use, it can make sense to keep it all in its own app directory. You can use `fly.toml` and `Dockerfile` like a normal app. + +The base image will stick around in the registry and as long as an image is associated with a Machine -- even if the Machine is stopped -- it won't be deleted. + +You can find the registry URL of the images for your base image in the dashboard under the **Registry** tab for the app, or by running `fly releases --image` + +## Related use case + +Want to skip the Fly builder and use Docker tools directly to manage your images? See [Managing Docker Images with Fly.io's Private Registry](https://fly.io/docs/blueprints/using-the-fly-docker-registry/), a guide that walks through building, pushing, deploying, and managing images via Fly.io’s Docker registry! + +## Related reading + +- [Deploy with a Dockerfile](https://fly.io/docs/languages-and-frameworks/dockerfile/) The general guide on how to deploy apps from Dockerfiles — useful context when you’re building a base image. +- [App Configuration (`fly.toml`)](https://fly.io/docs/reference/configuration/) Details on how your `fly.toml` orchestrates builds, images, and deployments. diff --git a/blueprints/using-the-fly-docker-registry.html.md b/blueprints/using-the-fly-docker-registry.html.md new file mode 100644 index 0000000000..38384d39e6 --- /dev/null +++ b/blueprints/using-the-fly-docker-registry.html.md @@ -0,0 +1,129 @@ +--- +title: Managing Docker Images with Fly.io's Private Registry +layout: docs +nav: guides +--- + +
+ Illustration by Annie Ruygt of Frankie the balloon and the Docker whale playing a game of bean bag toss +
+ +## Overview + +Fly.io lets you deploy Docker containers as lightweight Firecracker VMs, running globally. While `fly launch` supports many frameworks and auto-generates deployment configurations, you can also build and push your own Docker images to Fly.io’s private registry and deploy them manually. + +This guide walks through building, pushing, deploying, and managing images via Fly.io’s Docker registry. + +## Tagging images for the Fly.io registry + +Each Fly.io app gets a dedicated registry path in the form of: + +``` +registry.fly.io/ +``` + +To create a new app: + +```sh +fly apps create +``` + +App names must be globally unique. You can generate a unique one automatically: + +```sh +fly apps create --generate-name +``` + +Then tag your image for the app's registry: + +```sh +docker build -t registry.fly.io/: . +``` + +## Authenticating and pushing to the registry + +Authenticate Docker to use the Fly.io registry: + +```sh +fly auth docker +``` + +This command updates your `~/.docker/config.json` with the required auth token for `registry.fly.io`. Once authenticated, push your image: + +```sh +docker push registry.fly.io/: +``` + +## Deploying the image + +You can deploy images from the Fly.io registry or public registries, depending on where the image is hosted. + +### To deploy a private image pushed to Fly.io's registry: + +Use the `--image` option with the `fly deploy` command: + +```sh +fly deploy --app --image registry.fly.io/: +``` + +### To deploy a public image from Docker Hub or another public registry: + +Specify the image in your `fly.toml` file: + +```toml +[build] + image = "flyio/hellofly:latest" +``` + +## Reusing images across apps + +Although registry URLs are structured per app, access is scoped per organization. This means you can push an image to `registry.fly.io/app-1` and use it in another app (e.g. `app-2`) if both apps are in the same Fly.io organization: + +```sh +fly deploy --app app-2 --image registry.fly.io/app-1:tag +``` + +This pattern is useful for SaaS platforms that build once and deploy many times. + +## Verifying image existence + +You can verify that an image exists in the Fly registry without deploying it using Docker: + +```sh +docker manifest inspect registry.fly.io/my-app-name:my-tag +``` + +This command returns data if the image exists, or exits with code 1 if it does not. It requires prior authentication using `fly auth docker`. + +For most workflows, use the `fly deploy --image` option with the Fly Docker registry. For advanced flows like multi-app SaaS provisioning, track image tags using `docker manifest inspect` or associate them with Machines to retain them. + +## Listing image tags + +There is no API or GraphQL query to list all image tags in the registry. Tags only become visible to Fly when they are part of a release. You can view deployed image references with: + +```sh +fly releases -a my-app-name --image +``` + +If you're using the GraphQL API to query releases, use the following query format: + +```json +{ + "query": "query($appName: String!, $limit: Int!) { app(name: $appName) { releases: releasesUnprocessed(first: $limit) { nodes { id version description reason status imageRef stable user { id email name } createdAt } } } }", + "variables": { + "appName": "my-app-name", + "limit": 25 + } +} +``` + +This will only return images used in past deploys. + +## Related use case + +Want to optimize your deploys with shared layers and pre-installed dependencies? See the [Using base images for faster deployments](https://fly.io/docs/blueprints/using-base-images-for-faster-deployments/#how-to-make-a-base-image+external) blueprint. + +## Related reading + +- [Deploy with a Dockerfile](https://fly.io/docs/languages-and-frameworks/dockerfile/) The general guide on how to deploy apps from Dockerfiles — useful context when you’re building a base image. +- [App Configuration (`fly.toml`)](https://fly.io/docs/reference/configuration/) Details on how your `fly.toml` orchestrates builds, images, and deployments. diff --git a/blueprints/volume-forking.html.md b/blueprints/volume-forking.html.md new file mode 100644 index 0000000000..9c3c605c7c --- /dev/null +++ b/blueprints/volume-forking.html.md @@ -0,0 +1,130 @@ +--- +title: Using Fly Volume forks for faster startup times +layout: docs +nav: guides +author: kcmartin +date: 2025-09-12 +--- + +
+ Illustration by Annie Ruygt of a blimp and a Fly rocket flying side by side +
+ +
+**This guide shows how to preload large files onto volumes using forking to improve startup performance. This is especially useful for apps with big model files, binaries, or databases.** +
+ +## Overview + +If your app needs large local files like ML models, SQLite databases, or game assets, you're probably copying them into place at boot. Maybe you bake them into your image, or have your app download them on first run. This works, but it has downsides: + +- **Slow startup times**: waiting for multi-GB file downloads. +- **Wasted bandwidth**: each new machine pulls the same data. +- **Image bloat**: large files increase your OCI/Docker image size. + +There's a better way. Write those files to a Fly Volume so they _persist_ after they've been downloaded. Fly Volumes support [forking](docs/volumes/volume-manage/#create-a-copy-of-a-volume-fork-a-volume), a fast, storage-efficient way to create new volumes preloaded with data, skipping downloads and startup delays. + +--- + +## Why use this pattern? + +You’re cloning machines and want them to boot fast, with all their large files already in place. + +This isn’t about long-term persistence (though volumes can be used for that, too). It’s about getting cold-start performance close to warm-start, by skipping the “download half the internet” phase of your boot process. + +It’s especially useful when: + +- You’ve already run a setup step once and don’t want to repeat it. +- Your app reads large files but doesn’t modify them. +- You're scaling up on demand and startup time matters. + +--- + +## How volume forking works + +When you fork a volume: + +- **Data is copied from the source volume**, block-for-block. +- **Forked volumes are usable immediately**, even while they are in the hydrating state, thanks to lazy fetching (a system where data blocks are only fetched from the source volume when your app actually tries to read them). + +Your app can start reading files as soon as the fork succeeds. We start background replication right away, but behind the scenes we use a network block device to fetch blocks on demand. So startup is fast, and the data just shows up when your app asks for it. + +One tradeoff: disk access might be a little slower while the replication is ongoing. But for many apps, this beats the startup delay of downloading or unpacking files. + +--- + +## Example: cloning a machine with preloaded data + +### 1. Make sure the source volume you want to fork is available + +``` +fly volumes list -a your-app +``` + +Pick the volume with the data you want to reuse. + +### 2. Fork the volume + +``` +fly volumes fork vol_abc123 -a your-app +``` + +You’ll get output like this: + +``` + ID: vol_new_id + Name: volume_name + App: your-app + Region: yul + Zone: b5a1 + Size GB: 20 + Encrypted: true + Created at: 05 Sep 25 21:52 UTC +``` + +Note the `vol_new_id`. + +### 3. Clone the machine and attach the forked volume + +``` +fly machine clone --attach-volume vol_new_id +``` + +Now your new machine starts with the same data in place—no need for your app to download or generate it again. + +--- + +## What to know + +- **Forks aren’t instant behind the scenes.** You can read from them immediately, but the full copy happens in the background. Disk I/O might be slower until replication finishes. +- **Data is a snapshot, not a live link.** Changes to the original after forking don’t show up in the copy (and vice versa). +- **You can fork cross‑region.** The `--region` flag allows you to target another region for the fork. If you skip it, the volume ends up in the same region by default. Cross-region forks are expected to hydrate more slowly, since all data must be replicated over the network between regions. +- **Forks have storage costs.** As blocks are fetched or written, forked volumes grow. Keep an eye on storage usage costs if you fork a lot. + +--- + +## (Optional) Automate it in CI + +If you're scripting deploys or scaling machines dynamically, you can fold this pattern into a CI job or automation script. + +Here’s a rough sketch: + +``` +# Step 1: Fork the seed volume +FORKED_VOL=$(fly volumes fork vol_abc123 -a your-app | grep ID | awk '{ print $2 }') + +# Step 2: Clone the machine and attach the fork +fly machine clone --attach-volume $FORKED_VOL +``` + +This lets you create and attach preloaded volumes as part of your rollout process which is useful if you're standing up a new VM per job, or if you're scaling inference machines on demand. + +This is totally optional. Most folks will do this manually, or only once per release. But if you're already scripting `fly machine` operations, this fits right in. + +## Related reading + +- [Fly Volumes overview](/docs/volumes/overview/) +- [Create and manage volumes](/docs/volumes/volume-manage/) +- [Volume states](/docs/volumes/volume-states/) +- `fly volumes fork` [reference](/docs/flyctl/volumes-fork/) + diff --git a/blueprints/work-queues.html.md b/blueprints/work-queues.html.md new file mode 100644 index 0000000000..df43944ada --- /dev/null +++ b/blueprints/work-queues.html.md @@ -0,0 +1,181 @@ +--- +title: Deferring long-running tasks to a distributed work queue +layout: docs +nav: guides +author: kcmartin +date: 2025-07-17 +--- + +
+ Illustration by Annie Ruygt of a robot setting up a queue of boxes and tasks +
+ +## Overview + +Some things just take a while. Generating a PDF. Calling a slow third-party API. Running a machine learning model. Sending an email newsletter, and retrying if the mail server flakes out. + +Whatever the reason, if your web app is sitting around waiting for that work to finish, you're doing it wrong. Not just in the “inefficient” sense—but in the “your app will time out and confuse users and make debugging miserable” sense. + +Instead, you should offload that work to a background worker. + +This isn’t a new idea. The pattern’s been around forever, often called a **work queue,** **job queue,** or **task queue.** Here’s the gist: + +- The web app gets a request. +- Instead of doing everything right away, it drops a job onto a queue. +- A separate process—called a worker—pulls jobs from that queue and handles them in the background. + +This means your web handler can respond immediately. It also means your background work can run on different machines, retry on failure, and scale independently from your frontend. + +You’ve probably already used something like this without realizing it. Ever sent an email via a task queue? That was a worker. Ever seen a spinner on a dashboard while a report gets generated? There's probably a job queue behind it. + +In this guide, we’ll show you how to set this up—using Celery if you're in Python, BullMQ if you're in Node, and Fly Machines if you're allergic to running idle containers. + +If you're already comfortable with queues, skip ahead. But if you’ve been stuffing async tasks into HTTP handlers and hoping for the best, this is your exit ramp. Let’s clean it up. + +--- + +## Why use a work queue? + +- **Responsiveness**: Return HTTP responses instantly. Let workers take their time. +- **Scalability**: Scale web servers and workers independently. No need to waste compute on idle async calls. +- **Retries**: Jobs can fail and retry cleanly, without blocking a request or annoying a user. +- **Cost efficiency**: With Fly Machines, you only pay for worker time while the job is running. +- **Persistence**: Jobs don’t care if a user closes their browser tab. Work still gets done. + +--- + +## Redis and Valkey + +Most of the tools in this guide use [Redis](https://redis.io/about/) as a message broker—a fast, in-memory data store that's great for short-lived messages and task queues. If you're deploying on Fly.io, we recommend trying [Valkey](https://valkey.io/), a fork of Redis with a more active community and governance model. You can set up a Valkey cluster without needing an Enterprise license. It works the same way but gives you more flexibility and long-term stability. You can deploy either Redis or Valkey yourself or use a managed service like [Upstash Redis](/docs/upstash/redis/). + +--- + +## Option 1: Celery (Python) + +[Celery](https://docs.celeryq.dev/en/latest/getting-started/introduction.html) is a full-featured distributed task queue commonly used with Django and Flask. + +### How it works + +1. Your app enqueues a task to a message broker (Redis or RabbitMQ). +1. A worker process picks up the task from the queue and executes it. +1. Task results can optionally be stored (e.g., in Redis or your database). + +### Setup + +Install the basics: + +```bash +pip install "celery[redis]" django-celery-results +``` + +Create a `celery.py`: + +```python +from celery import Celery + +app = Celery("my_app") +app.config_from_object("django.conf:settings", namespace="CELERY") +app.autodiscover_tasks() +``` + +Use `@shared_task` for defining tasks: + +```python +from celery import shared_task + +@shared_task(bind=True, autoretry_for=(Exception,), retry_backoff=3, retry_kwargs={"max_retries": 5}) +def crunch_numbers(self, arg): + ... +``` + +Run your worker: + +```bash +celery -A my_app worker -l info +``` + +### Deploying to Fly.io + +Use `fly launch` to generate config and add Upstash Redis. + +Update `fly.toml`: + +``` +[processes] +app = "gunicorn my_app.wsgi" +worker = "celery -A my_app worker -l info" +``` + +Then: + +```bash +fly deploy +fly scale count worker=2 +``` + +
**Region tip**: Put your Redis instance and workers in the same region (e.g. `ord`, `fra`) to avoid unnecessary latency and weird race conditions.
+ +--- + +## Option 2: On-Demand Workers with Fly Machines + +You don’t always need a persistent queue and worker pool. For low-frequency or one-off jobs, you can spin up a Fly Machine per task. + +### How it works + +1. Your app writes a job (params, metadata) to Redis/Valkey. +1. It calls the Fly Machines API to spin up a temporary worker. +1. The worker reads job data from Redis, runs it, and writes back the result. +1. Your app polls Redis to check status and fetch results. +1. You shut down the machine and clean up. + +This is great for jobs that don’t happen often, and it keeps your infrastructure lean. You can reuse your app image or build a dedicated worker image. + +Just make sure your job data and results are JSON-serializable. + +--- + +## Option 3: BullMQ (Node.js) + +BullMQ is a Redis-backed queue system for Node.js apps. It works similarly to Celery: + +- Your app enqueues jobs to Redis/Valkey. +- Workers pull jobs and execute them. +- You get features like retries, rate limits, job delays, and concurrency control. + +BullMQ plays nicely with TypeScript, has good docs, and integrates well with modern backend frameworks like NestJS. + +Provision Redis via Upstash or use `fly redis create`. + +Scale workers as needed based on job load and machine size: + +```javascript +const worker = new Worker("my-queue", async job => { + // do your work here +}, { concurrency: 4 }); +``` + +--- + +## Other Options + +If none of the above tools fit your stack or mental model, there are plenty of alternatives: + +- **Sidekiq** (Ruby): The de facto standard for background jobs in Ruby and Rails apps. Redis-backed, battle-tested, and fast. +- **Oban** (Phoenix) A robust background job processing library for Elixir +- **RQ** (Python): A simpler, more lightweight job queue than Celery, great for smaller projects or simpler needs. + +Most of these integrate with Redis/Valkey, some support more exotic brokers (like Postgres or Kafka), and all of them can be deployed on Fly.io with the right setup. + +--- + +## Final Notes + +Whatever you do, don’t handle long-running jobs inside your HTTP request handlers. Use a job queue. Pick the right tool for your stack and use case, and let your web server get back to doing what it’s good at—responding fast and not melting down under load. + + + +## Related reading + +- [Async workers on Fly Machines](https://fly.io/blog/python-async-workers-on-fly-machines/) +- [Celery and Django for async tasks](https://fly.io/django-beats/celery-async-tasks-on-fly-machines/) diff --git a/build/assets/manifest-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.js b/build/assets/manifest-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.js new file mode 100644 index 0000000000..e69de29bb2 diff --git a/build/assets/manifest-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.js.gz b/build/assets/manifest-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.js.gz new file mode 100644 index 0000000000..66160efeab Binary files /dev/null and b/build/assets/manifest-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.js.gz differ diff --git a/build/assets/manifest.json b/build/assets/manifest.json new file mode 100644 index 0000000000..959ddc75a3 --- /dev/null +++ b/build/assets/manifest.json @@ -0,0 +1 @@ +{"files":{"manifest-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.js":{"logical_path":"manifest.js","mtime":"2025-05-15T14:33:13-07:00","size":0,"digest":"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855","integrity":"sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU="}},"assets":{"manifest.js":"manifest-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.js"}} \ No newline at end of file diff --git a/caching/full-stack.html.md b/caching/full-stack.html.md deleted file mode 100644 index 3487dcc38b..0000000000 --- a/caching/full-stack.html.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Full Stack Caching -layout: docs -sitemap: false -nav: firecracker ---- - -Frameworks for full stack apps typically include powerful caching APIs. Developers can use these to cache everything from full HTTP responses to view partials to structured data. In general, the goal of caching in full stack apps is improve performance and levers for scaling. - -Users prefer fast applications regardless of scale. The standard caching APIs in frameworks give you a way to make your application fast without spending wasted time prematurely optimizing some of the more finicky bits of your app. Rather than over optimizing a DB schema that will probably change as you iterate, for example, you can cache a rendered view and avoid hitting the database altogether. - -Responsiveness is usually a function of latency. When a user requests data, applications typically deal with: - -* Input lag, how long it takes something they type/clip/tap/say to appear on the screen. If your app is simple you hopefully don't have to worry about this. If you put in a load of third party JavaScript, it might be a problem. -* The "slow" speed of light: when a user sends data to your backend, they have to wait for a light to go from their device, through their ISP (which can be frightfully slow), across the world, and into your server. -* Data processing time: you probably have to do some computering to handle a user's request and then generate a response. -* The speed of light again! Your server sends data, it traverses the internet, arrives at the users (frightfully slow) ISP and lands on their device. -* Render lag: there's more computering involved when a device paints information on a user's screen. - -Humans are reasonably sensitive to delays. Most people can perceive a slow down of as little as 100ms. The best apps have a latency budgets -- a "deadline" for responding to user input. This can vary a bit depending on the action, if a user makes a complicated request and gets a message that "we're working on it" very quickly, they might tolerate several seconds of delay. If they do something that seems simple (like click a URL on Google), they tend to be much less forgiving. - -Latency budgets are hard! You can really only control two things: (1) render performance and (2) how fast you can get data to a user's ISPs. - -Fly helps you get data to a user's ISP faster by running your app geographically near your users. A user who makes a request to your app in San Francisco, for example, will connect to a server running in San Jose, usually in under 5ms. If that same San Francisco user connects to New York City to use your application, it might take more like 150ms. Remember the 100ms "perceptible to humans" number? Light is so "slow" a human can perceive how long it takes to travel across the United States. - -So if your app just does some math without talking to a DB or an outside service, Fly is great! But apps that do that are boring. Your users probably care about data you manage on their behalf. If you're using a typical database, it's difficult and expensive to move close to users ... simplest to keep it in one place. So putting your app in different cities reduces latency between your users and the application logic, but might increase latency between your application logic and the data it uses to do magic. - -The trick is to avoid requesting data from your primary database. And when you do request "slow" data, to keep it in a fast cache store so you can skip that query next time. - -Fly hosted applications include local Redis cache storage. When you request data from your database, you can save the response (or a transformed version of the response) in the cache. The next time you request that data, you'll get it back in just a few milliseconds. Most framework cache APIs can use Redis, so if you enable caching in your app you'll see an immediate performance improvement on Fly with minimal code. - -## _View caching_ - -Most popular frameworks offer Redis adapters for view caching. If you’ve configured view caching already, you can typically just add a Redis adapter library, and point at [Fly’s Redis cache](/docs/redis/). - - -If your application purges the caches when data changes, you’ll need to add a little bit of logic to [send the purge/delete commands to `database 1`](/docs/redis/#managing-redis-data-globally) on your Fly Redis connection. This ensures they get applied in every region. diff --git a/database-storage-guides.html.md b/database-storage-guides.html.md index 82720aadd5..e25d8ca969 100644 --- a/database-storage-guides.html.md +++ b/database-storage-guides.html.md @@ -1,65 +1,37 @@ --- -title: Databases & Storage +title: Databases and storage layout: docs toc: true nav: firecracker --- -## Choosing an approach to storage +
+ Illustration by Annie Ruygt of a tiger sleeping next to a storage device +
-Often the solution to persistent data storage is to connect your Fly App to a separate database or object store. -Fly.io offers a [deploy-it-yourself Postgres app](/docs/postgres/) with some tools to make it easier to manage yourself. +## Managed database service -If you need hardware-local disk storage on your Machines—for example, if your Fly App _is_ a database (or if you want to use [LiteFS](/docs/litefs))—then you'll want to use [Fly Volumes](/docs/volumes/). +[Fly.io Managed Postgres](/docs/mpg/) is a production-ready Postgres service that handles the hard parts for you: high availability, automatic failover, encrypted backups, monitoring and metrics, seamless scaling, and 24/7 support to keep your data fast, safe, and always online. -Explore these, and further options, for data storage in the following sections. +## Key-value stores -## Fly Volumes - Disk Storage +**[Upstash for Redis](/docs/upstash/redis/)** - [Redis](https://redis.io/+external) is an in-memory database commonly used for caching. A managed service by [Upstash](https://upstash.com/+external). -Anything an App VM writes to its root disk is ephemeral: when the VM is redeployed, the root file system is rebuilt using its Docker image, deleting any data written to it since it was started up. This is fine for `/tmp` files, but most apps need to keep state in a database or another form of persistent storage. - -- **[Fly Volumes](/docs/volumes/)** - Persistent storage on Fly.io is provided by Fly Volumes. You can use Volumes on an App directly, or run a separate database App with Volume storage and connect an App to that. A Fly Volume is a slice of NVMe disk storage attached to the server that hosts your Machine. Volumes have pros and cons, and you should read the [Fly volumes overview](/docs/reference/volumes/) before deciding whether they're the best solution for your use case. - -## Fly.io Database Projects - -These are projects that we develop and support. They're not managed services; you can deploy them yourself as Fly Apps. - -- **[Fly Postgres](/docs/postgres/)** - [PostgreSQL](https://www.postgresql.org/) is a popular relational database. When you deploy an App on Fly.io, we give you the option to launch a Fly Postgres App and attach it to your App. We provide tools for deployment and management; you manage the cluster once it's deployed. -- **[LiteFS for SQLite](/docs/litefs/)** - [SQLite](https://www.sqlite.org/index.html) is a very lightweight file-based database. [LiteFS](/docs/litefs/) is a distributed file system that transparently replicates SQLite databases. You deploy it and you manage it. - -## Partner Integrations Running on Fly.io - -- **[Upstash for Redis](/docs/reference/redis/)** - [Redis](https://redis.io/) is an in-memory database commonly used for caching. A managed service by [Upstash](https://upstash.com/). - -## Other Storage Apps - -If your application calls for a different solution, you can deploy it yourself as an App on Fly.io. These examples can help you get started with other popular storage options. - -- **[MySQL and MariaDB](/docs/app-guides/mysql-on-fly/)** - [MySQL](https://www.mysql.com/) is a popular relational database. [MariaDB](https://mariadb.org/) is a community fork of MySQL and is compatible with MySQL. +--- -- **[EdgeDB](/docs/app-guides/edgedb/)** - [EdgeDB](https://www.edgedb.com/) is a graph-relational database that runs on top of Postgres. +## Fly Volumes - Disk storage -- **[MinIO Object Storage](/docs/app-guides/minio/)** - [MinIO](https://min.io/) is software that allows you to self-host S3-compatible storage. +The Fly Machines in your app provide ephemeral storage, so you get a blank slate on every startup. For hardware-local, persistent storage on Fly.io, use Fly Volumes. You can attach volumes on an app directly, or run a separate database app with volume storage and connect an app to that. -## Recommended External Providers +**[Fly Volumes](/docs/volumes/overview):** A Fly Volume is a slice of NVMe disk storage attached to the server that hosts your Machine. Read the [Fly Volumes overview](/docs/volumes/overview/) to find out if volumes are the best solution for your use case. -If you want a fully managed database or storage solution for your Fly Apps, there are many great options, including: +## Object storage service -- [Crunchy Bridge Managed Postgres](https://www.crunchydata.com/products/crunchy-bridge) (on AWS, Azure, GCP, or Heroku) -- [Neon Serverless Postgres](https://neon.tech/) -- [PlanetScale Serverless MySQL](https://planetscale.com/) ([guide to use with Fly Apps](/docs/app-guides/planetscale/)) -- [Supabase Postgres](https://supabase.com/database) -- [MinIO Hosted Object Storage](https://min.io/) -- [Fauna](https://fauna.com/) ([guide to use with Fly Apps](/docs/app-guides/fauna/)) +**[Tigris Global Object Storage](/docs/tigris/)** - [Tigris](https://www.tigrisdata.com/+external) is a globally distributed S3-compatible object storage service hosted on Fly.io infrastructure. -## Other Places +--- -You can connect your Fly Apps to the usual suspects, too: +## Have a project with special database or storage requirements? -- [Amazon RDS for PostgreSQL](https://aws.amazon.com/rds/postgresql/) -- [Azure Database for PostgreSQL](https://azure.microsoft.com/en-us/products/postgresql/#overview) -- [Digital Ocean Managed Postgres](https://www.digitalocean.com/products/managed-databases-postgresql) -- [Google Cloud SQL for PostgreSQL](https://cloud.google.com/sql/docs/postgres/) -- [Heroku Managed Data Services](https://www.heroku.com/managed-data-services) -- [AWS S3 Object Storage](https://aws.amazon.com/s3/) +If your project calls for something else, most external databases and storage providers work well with Fly.io. You might just need to spend a bit more time wiring up the networking. diff --git a/deep-dive/application.html.markerb b/deep-dive/application.html.markerb new file mode 100644 index 0000000000..3eb912216e --- /dev/null +++ b/deep-dive/application.html.markerb @@ -0,0 +1,22 @@ +--- +title: Your Fly App +layout: docs +nav: demo +order: 4 +--- + +
+ Illustration by Annie Ruygt of an app cube flying near two Fly balloons +
+ +Your demo app is running by default on two [Fly Machines](/docs/machines/), our fast-launching VMs. + +Both Machines stop when not in use, and start automatically when a new request comes in. [Autostop/autostart](/docs/launch/autostop-autostart/) is entirely configurable. You can chose to suspend instead of stop, configure a minimum number of Machines to keep running, or even decide never to stop Machines at all. + +The purpose of two Machines is twofold: redundancy and scalability. If one Machine goes down, the other can continue on. If both are available, when your app has higher traffic, both can be started to handle requests. You can [vertically scale](/launch/scale-machine/) the CPU and RAM on Machines. + +You can also [horizontally scale](/docs/launch/scale-count/) the number of Machines. You can scale out to different regions with the `fly scale count` command too. If you have a co-worker on another continent, create a Machine there. + +Familiarize yourself with the [fly.toml](/docs/reference/configuration/) config file. Make a change there--or to any part of your app--and run `fly deploy` to update your app. + +Your Fly App takes advantage of [Anycast routing](/docs/networking/services/), a [load-balancing proxy](/docs/reference/fly-proxy/), and [private networking](/docs/networking/private-networking/). You can also create [DNS certificates for custom domains](/docs/networking/custom-domain/). diff --git a/deep-dive/django.html.markerb b/deep-dive/django.html.markerb new file mode 100644 index 0000000000..c7f6a6ef2b --- /dev/null +++ b/deep-dive/django.html.markerb @@ -0,0 +1,43 @@ +--- +title: Django demo reference +layout: docs +nav: demo +--- + +The Django source is on [GitHub](https://github.com/fly-apps/django-dictaphone/). +`fly launch` will provide a [`Dockerfile`](https://docs.docker.com/reference/dockerfile/) + and a [`fly.toml`](/docs/reference/configuration/) config file. + When you make changes to your application, you can use + [dockerfile-django](https://github.com/fly-apps/dockerfile-django?tab=readme-ov-file#dockerfile-generator-for-django-projects) +to produce updated Dockerfiles. + +How the pieces are put together: + +* The original web-dictaphone app (minus the HTML) is in the + [static](https://github.com/fly-apps/django-dictaphone/tree/main/static) directory, + and contains icons, scripts, styles, and a web manifest; all served as + static files. +* [`clips/templates/clips/index.html`](https://github.com/fly-apps/django-dictaphone/blob/main/clips/templates/clips/index.html) contains the HTML template. +* [Django Models and databases](https://docs.djangoproject.com/en/5.1/topics/db/) is [configured](https://github.com/fly-apps/django-dictaphone/blob/2a9ec4ceb728e657aeec76477d87ea1a8627523a/dictaphone/settings.py#L89-L102) to use Sqlite3 as the database in development mode, and + PostgreSQL in production. Access to the database is through the `DATABASE_URL` secret. +* [Django Storages](https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html) is [configured](https://github.com/fly-apps/django-dictaphone/blob/2a9ec4ceb728e657aeec76477d87ea1a8627523a/dictaphone/settings.py#L104-L117) to write to the filesystem + in development and Tigris in production. The following secrets are used to + establish the connection: `AWS_ACCESS_KEY_ID`, `AWS_ENDPOINT_URL_S3`, `AWS_REGION`, `AWS_SECRET_ACCESS_KEY`, and `BUCKET_NAME`. +* [Django Channels](https://channels.readthedocs.io/en/latest/) is [configured](https://github.com/fly-apps/django-dictaphone/blob/2a9ec4ceb728e657aeec76477d87ea1a8627523a/dictaphone/settings.py#L119-L135) to use an in memory channel in development and Redis in production + based using the `REDIS_URL` secret. + +Key points of logic: + +* [`static/scripts/app.js`](https://github.com/fly-apps/django-dictaphone/blob/main/static/scripts/app.js) + has been modified to make server requests at various points using + [the Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch). +* [`clips/views.py`](https://github.com/fly-apps/django-dictaphone/blob/main/clips/views.py) contains + the logic to build responses to requests for the index page, and to GET, PUT, + and DELETE audio clips. +* The server side realtime implementation code is primarily in [Django Channels](https://channels.readthedocs.io/en/latest/), so all that is needed is a few lines in [`clips/models.py`](https://github.com/fly-apps/django-dictaphone/blob/b1bb70cc513bd4977dc74fd56dfff86b8f2d12ec/clips/models.py#L20-L25). + The client side is provided by [`static/scripts/websocket.js`](https://github.com/fly-apps/django-dictaphone/blob/main/static/scripts/websocket.js). + When update notifications are received, the index page is re-retrieved and the body of the DOM is replaced with new contents. +* When the `WHISPER_URL` secret is set, `PUT` requests will cause the audio clips + to be passed to the Whisper server, and responses will be used to update the + PostgreSQL database. This is done using [Celery](https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html). + The code for this is in [`clips/tasks.py`](https://github.com/fly-apps/django-dictaphone/blob/main/clips/tasks.py). \ No newline at end of file diff --git a/deep-dive/index.html.markerb b/deep-dive/index.html.markerb new file mode 100644 index 0000000000..3e4ff3495f --- /dev/null +++ b/deep-dive/index.html.markerb @@ -0,0 +1,74 @@ +--- +title: Deep dive demo +layout: docs +nav: demo +toc: true +--- + +
+ Illustration by Annie Ruygt of a bird in a diver costume, swimming under water +
+ +Welcome to our deep dive demo, where you can explore Fly.io more thoroughly, but in a time-boxed way. In one hour or less: get a fully-functioning app running in the first few minutes, and then have enough time left over to understand what you just did, explore how the pieces fit together, and even integrate AI functionality that makes use of GPUs. + +## Goals of the deep dive + +It's really just one goal: + +**Help you find out if Fly.io is the right place for you.** + +If you came from our [Speedrun](https://fly.io/speedrun) or [Getting started](https://fly.io/docs/getting-started/) pages, maybe you want to go beyond `hello world` and explore the platform so you can feel confident that you're choosing a provider that can support you now, and scale with you later on as you grow. + +Maybe you're concerned about lock in. You want to use services from other places and even eject entirely and go elsewhere if things don't work out. + +Perhaps it is a simple as you feel it is easier to tune and evolve a working +system than it is to debug why your application doesn’t launch on an unfamiliar +platform. + +If you don't get a good feeling within an hour, you're out of here. If you do get a good feeling, then stick around and try launching the app that you brought here to launch. + +## Beyond `hello fly` + +A real-world application has, at a minimum, the following components: + +* An HTML form and a database; typically a relational database. +* The ability to handle media files or documents, generally using S3. +* A multi-user and realtime component, where changes made by one person in one location are reflected instantly in the browser of another person. + +To get a fully-functional app running smoothly, you need a whole bunch of things; things like Anycast routing, load balancers, DNS certificates, WebSockets, an internal private network, a relational database, an object store, and an in-memory database. And the knowledge to connect them all together. + +Connecting complex components together generally takes an unpredictable amount of time due to surprises. Usually, set up for a complete app typically takes a _minimum_ of an afternoon's worth of work, even if you're familiar with a cloud platform. We will be up and running in minutes. + +Fly.io minimizes surprises by taking a lot of work off your plate with our [Public Network Services](https://fly.io/docs/networking/services/), [Fly Proxy routing](/docs/reference/fly-proxy/), out-of-the-box [Private Networking](/docs/networking/private-networking/), and preconfigured databases. +While we can't promise no surprises, we can show you how we'll partner with you to handle some of the dev ops complexities of working in a public cloud. + +## So what is this deep dive demo app? + +The deep dive demo uses industry standard components that you can run on your laptop, a VPS, AWS EC2, Google Compute Engine, or Azure. + +### Web Dictaphone -- Fly.io edition + +The deep dive demo is based on [MDN's Web Dictaphone](https://github.com/mdn/dom-examples/tree/main/media/web-dictaphone+external). +You can play with a [live demo hosted on GitHub](https://mdn.github.io/dom-examples/media/web-dictaphone/+external). The Web Dictaphone app is about as basic of an HTML form as you can get, and it has the added bonus of providing the ability to generate as many media files as you want using only your voice. + +The basic Web Dictaphone is client side only, requiring a web server that can deploy static assets (HTML, CSS, JS, images), like nginx, Apache HTTPd, or Caddy. Storing the data in databases for our deep dive demo requires a server that can handle HTTP GET, POST, PUT, and DELETE requests. The choice of server varies depending on the language or framework. + +The demo includes the following components: + +* A [PostgreSQL](https://www.postgresql.org/+external) relational database to store the names of the audio clips +* A [Tigris bucket](https://www.tigrisdata.com/+external) to store the audio files +* [Upstash for Redis](https://fly.io/docs/reference/redis/) and [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API+external) to handle the connections for the realtime requirement + +### Available runtimes + +Our deep dive demo comes in five flavors (so far): + + * [Django](/docs/deep-dive/django/) + * [Next.js](/docs/deep-dive/nextjs/) + * [Node.js](/docs/deep-dive/nodejs/) + * [Phoenix](/docs/deep-dive/phoenix/) + * [Rails](/docs/deep-dive/rails/) + +--- + +**Next:** [Launch the demo](/docs/deep-dive/launch-deep-dive/) diff --git a/deep-dive/launch-deep-dive.html.markerb b/deep-dive/launch-deep-dive.html.markerb new file mode 100644 index 0000000000..379e1bb496 --- /dev/null +++ b/deep-dive/launch-deep-dive.html.markerb @@ -0,0 +1,66 @@ +--- +title: Launch the demo +layout: docs +nav: demo +order: 3 +--- + +Let's get right to it. + +**Step 0:** To get set up to run the deep dive demo on Fly.io, all you need to do is install Node, Phoenix, Python, or Ruby and create an empty project directory. + +After that there are only two steps. + +**Step 1:** [Install flyctl](https://fly.io/docs/flyctl/install/). + +**Step 2:** Run one of the following: +```cmd +fly launch --from https://github.com/fly-apps/django-dictaphone +``` +-or- +```cmd +fly launch --from https://github.com/fly-apps/node-dictaphone +``` +-or- +```cmd +fly launch --from https://github.com/fly-apps/nextjs-dictaphone +``` +-or- +```cmd +fly launch --from https://github.com/fly-apps/phoenix-dictaphone +``` +-or- +```cmd +fly launch --from https://github.com/fly-apps/rails-dictaphone +``` + +If you're new to Fly.io, the second step will take you to a page where you can register before continuing. + +The `fly launch` output describes what you'll be getting for the app, and gives you an opportunity to tweak the settings. (Suggestion: don't. The defaults are fine for this demo and we'll walk you through how to adjust them later.) And then it will build and assemble and wire up your app. + +Take your time and play with it. Open the application in multiple browser windows. Send a link to a friend on another continent and watch your browser update in realtime. + +And then relax. We promised you it would be less than an hour. You're already up and running. If you are so inclined, try bringing up this exact same application on another cloud provider. We don't mind. In fact, we encourage it. Just please don't count the time you spent there against the hour you set aside for this demo! + +## Explore the result of `fly launch` + +Once you're back and/or rested up, explore the app and add-on components. Feel free to review the following in any order, or chose to skip ahead: + + * [Your Fly App](../application/) + * [PostgreSQL database](../postgresql/) + * [Tigris Object Storage](../tigris/) + * [Upstash Redis](../redis/) + +## Runtimes + +Optional, but if you haven't yet dived in, details are provided for each runtime: + + * [Django](../django/) + * [Next.js](../nextjs/) + * [Node.js](../nodejs/) + * [Phoenix](../phoenix/) + * [Rails](../rails/) + +--- + +**Next:** Add [OpenAI Whisper](../whisper/) speech recognition as a bonus. diff --git a/deep-dive/nextjs.html.markerb b/deep-dive/nextjs.html.markerb new file mode 100644 index 0000000000..180f3cbade --- /dev/null +++ b/deep-dive/nextjs.html.markerb @@ -0,0 +1,55 @@ +--- +title: Next.js demo reference +layout: docs +nav: demo +--- + + + +The Next.js demo source is on [GitHub](https://github.com/fly-apps/nextjs-dictaphone). +`fly launch` will provide a [`Dockerfile`](https://docs.docker.com/reference/dockerfile/) + and a [`fly.toml`](https://fly.io/docs/reference/configuration/) config file. When you make changes to your application, you can run `npx @flydotio/dockerfile` to produce updated Dockerfiles. + +How the pieces are put together: + +* The original web-dictaphone app is placed in the [Next.js app](https://nextjs.org/docs/app) directory: + * [`app/dictaphone.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/app/dictaphone.js) + contains the JavaScript. + * [`app/globals.css`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/app/globals.css) contains + the stylesheets. + * [`app/page.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/app/page.js) contains the HTML +* [`server.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/server.js) and + [`app/audio/[name]/route.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/app/audio/%5Bname%5D/route.js) contain + the server implementation. +* The [pg](https://www.npmjs.com/package/pg) module is used to access PostgreSQL. + Access to the database is through the `DATABASE_URL` secret. This module is low level + which is sufficient for this demo, but most modern Node.js applications use + an [ORM](https://amplication.com/blog/top-6-orms-for-modern-nodejs-app-development). +* The [AWS SDK for JavaScript](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/welcome.html) + is used to access Tigris. The following secrets are used to + establish the connection: `AWS_ACCESS_KEY_ID`, `AWS_ENDPOINT_URL_S3`, `AWS_REGION`, `AWS_SECRET_ACCESS_KEY`, and `BUCKET_NAME`. +* The [express-ws](https://github.com/HenningM/express-ws?tab=readme-ov-file#express-ws-) + module is used for [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API) support. +* The [redis](https://github.com/redis/node-redis?tab=readme-ov-file#node-redis) module + is used to access Redis. The `REDIS_URL` secret is used to access the database. + +Key points of logic: + +* [`app/page.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/app/page.js) contains the + home page. All of the clip data comes from web sockets. +* [`app/dictaphone.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/app/dictaphone.js) + has been modified to make server requests at various points using + [the Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch); the delete + and rename actions are split out so that they can be referenced by the React JSX Home page view. +* [`server.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/server.js) contains + the WebSocket code, and delegates to Next.js to render the rest. +* [`pubsub.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/pubsub.js) contains the + interface to Redis. It is split out from the server to avoid circular dependencies. +* [`app/audio/[name]/route.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/app/audio/%5Bname%5D/route.js) contains + the logic to build responses to requests to GET, PUT, + and DELETE audio clips. +* [`app/page.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/ac1abcdc9cb1dfa7de08c682b26034e76ad34cc6/app/page.js#L12-L21) + contains the client implementation of web sockets. +* When the `WHISPER_URL` secret is set, `PUT` requests will cause the audio clips + to be passed to the Whisper server, and responses will be used to update the + PostgreSQL database. The code for this is in [`app/audio/[name]/route.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/459774ec9ebebdf6f991adaa6c88509854ffe34e/app/audio/%5Bname%5D/route.js#L90-L118). diff --git a/deep-dive/nodejs.html.markerb b/deep-dive/nodejs.html.markerb new file mode 100644 index 0000000000..ed10bb1dc0 --- /dev/null +++ b/deep-dive/nodejs.html.markerb @@ -0,0 +1,53 @@ +--- +title: Node.js demo reference +layout: docs +nav: demo +--- + + + +The Node.js demo source is on [GitHub](https://github.com/fly-apps/node-dictaphone). +`fly launch` will provide a [`Dockerfile`](https://docs.docker.com/reference/dockerfile/) + and a [`fly.toml`](https://fly.io/docs/reference/configuration/) config file. When you make changes to your application, you can run `npx @flydotio/dockerfile` to produce updated Dockerfiles. + +How the pieces are put together: + +* The original web-dictaphone app (minus the HTML) is in the + [public](https://github.com/fly-apps/node-dictaphone/tree/main/public) directory, + and contains icons, scripts, styles, and a web manifest; all served as + static files. +* [`views/index.ejs`](https://github.com/fly-apps/node-dictaphone/blob/main/views/index.ejs) contains the HTML template. +* [app.js](https://github.com/fly-apps/node-dictaphone/blob/main/app.js) contains + the server implementation, based on [Express.js](https://expressjs.com/). There + are lots of JavaScript frameworks out there, this is perhaps the oldest and most + minimalist. +* The [pg](https://www.npmjs.com/package/pg) module is used to access PostgreSQL. + Access to the database is through the `DATABASE_URL` secret. This module is low level + which is sufficient for this demo, but most modern Node.js applications use + an [ORM](https://amplication.com/blog/top-6-orms-for-modern-nodejs-app-development). +* The [AWS SDK for JavaScript](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/welcome.html) + is used to access Tigris. The following secrets are used to + establish the connection: `AWS_ACCESS_KEY_ID`, `AWS_ENDPOINT_URL_S3`, `AWS_REGION`, `AWS_SECRET_ACCESS_KEY`, and `BUCKET_NAME`. +* The [express-ws](https://github.com/HenningM/express-ws?tab=readme-ov-file#express-ws-) + module is used for [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API) support. +* The [redis](https://github.com/redis/node-redis?tab=readme-ov-file#node-redis) module + is used to access Redis. The `REDIS_URL` secret is used to access the database. + +Key points of logic: + +* [`public/scripts/app.js`](https://github.com/fly-apps/node-dictaphone/blob/main/public/scripts/app.js) + has been modified to make server requests at various points using + [the Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch). +* [`app.js`](https://github.com/fly-apps/node-dictaphone/blob/main/app.js) contains + the logic to build responses to requests for the index page, and to GET, PUT, + and DELETE audio clips. +* [`pubsub.js`](https://github.com/fly-apps/node-dictaphone/blob/main/pubsub.js) contains + the logic to subscribe to a `dictaphone:timestamp` redis queue, and to + forward updates to all open websocket connections. +* [`websocket.js`](https://github.com/fly-apps/node-dictaphone/blob/main/public/scripts/websocket.js) + contains the client implementation of web sockets. When update notifications + are received, the index page is re-retrieved and the body of the DOM is + replaced with new contents. +* When the `WHISPER_URL` secret is set, `PUT` requests will cause the audio clips + to be passed to the Whisper server, and responses will be used to update the + PostgreSQL database. The code for this is in [app.js](https://github.com/fly-apps/node-dictaphone/blob/1e84a4dece6888dfc68880d146b46511d47391b3/app.js#L102-L129). diff --git a/deep-dive/phoenix.html.markerb b/deep-dive/phoenix.html.markerb new file mode 100644 index 0000000000..7f2b68c7db --- /dev/null +++ b/deep-dive/phoenix.html.markerb @@ -0,0 +1,51 @@ +--- +title: Phoenix demo reference +layout: docs +nav: demo +--- + +The Phoenix demo source is on [GitHub](https://github.com/fly-apps/phoenix-dictaphone). +`fly launch` will provide a [`Dockerfile`](https://docs.docker.com/reference/dockerfile/) + and a [`fly.toml`](https://fly.io/docs/reference/configuration/) config file. + +How the pieces are put together: + +* The original web-dictaphone app is spread throughout your application: + * [`assets/js/dictaphone.js`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/assets/js/dictaphone.js) + contains the JavaScript. + * [`assets/css/dictaphone.css`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/assets/css/dictaphone.css) contains + the stylesheets. + * [`priv/static/favicon.ico`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/priv/static/favicon.ico) contains your image + * [`lib/dictaphone_web/live/dictaphone.html.heex`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/lib/dictaphone_web/live/dictaphone.html.heex) contains the HTML +* [`lib/dictaphone_web/live/dictaphone.ex`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/lib/dictaphone_web/live/dictaphone.ex) and + [`lib/dictaphone_web/controllers/audio_controller.ex`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/lib/dictaphone_web/controllers/audio_controller.ex) contain + the server implementation. +* [Ecto.Adapters.Postgres](Ecto.Adapters.Postgres) is used to access PostgreSQL. + Access to the database is through the `DATABASE_URL` secret. +* [ExAws.S3](https://hexdocs.pm/ex_aws_s3/ExAws.S3.html) + is used to access Tigris. The following secrets are used to + establish the connection: `AWS_ACCESS_KEY_ID`, `AWS_ENDPOINT_URL_S3`, `AWS_REGION`, `AWS_SECRET_ACCESS_KEY`, and `BUCKET_NAME`. +* [Phoenix.LiveView](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html) + module is used for [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API) support. +* [Phoenix.PubSub.Redis](https://hexdocs.pm/phoenix_pubsub_redis/Phoenix.PubSub.Redis.html) + is used to access Redis. The `REDIS_URL` secret is used to access the database. + +Key points of logic: + +* [`lib/dictaphone_web/live/dictaphone.html.heex`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/lib/dictaphone_web/live/dictaphone.html.heex) contains the + home page. +* [assets/js/dictaphone.js](https://github.com/fly-apps/phoenix-dictaphone/blob/main/assets/js/dictaphone.js) + has been modified to: + * upload clips using [the Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch) + * use [Click Events](https://hexdocs.pm/phoenix_live_view/bindings.html#click-events) for delete + * use [Client Hooks](https://hexdocs.pm/phoenix_live_view/js-interop.html#client-hooks-via-phx-hook) for rename actions + the WebSocket code, and delegates to Next.js to render the rest. +* [`lib/dictaphone_web/live/dictaphone.ex`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/lib/dictaphone_web/live/dictaphone.ex) contains the + interface to Redis. It is split out from the server to avoid circular dependencies. +* [`app/audio/[name]/route.js`](https://github.com/fly-apps/nextjs-dictaphone/blob/main/app/audio/%5Bname%5D/route.js) contains + the logic to respond to events from both browser clients, and redis pubsub. +* [`lib/dictaphone_web/controllers/audio_controller.ex`](https://github.com/fly-apps/phoenix-dictaphone/blob/main/lib/dictaphone_web/controllers/audio_controller.ex) + contains the server access to `GET` and `PUT` audio clips. +* When the `WHISPER_URL` secret is set, `PUT` requests will cause the audio clips + to be passed to the Whisper server, and responses will be used to update the + PostgreSQL database. The code for this is in [`lib/dictaphone_web/controllers/audio_controller.ex`](https://github.com/fly-apps/phoenix-dictaphone/blob/396d22a537b4f688f99ce88b707d6e045d828e6f/lib/dictaphone_web/controllers/audio_controller.ex#L42-L58). diff --git a/deep-dive/postgresql.html.markerb b/deep-dive/postgresql.html.markerb new file mode 100644 index 0000000000..745cfd3201 --- /dev/null +++ b/deep-dive/postgresql.html.markerb @@ -0,0 +1,16 @@ +--- +title: PostgreSQL +layout: docs +order: 5 +nav: demo +--- + +
+ Illustration by Annie Ruygt of elephant and a Fly balloon +
+ +Fly Postgres is deployed as a separate app, and that app comes initially configured with a single [Fly Postgres](/docs/postgres/) Machine. That's fine for development, but for production you need redundancy and scalability. With a [few commands](/docs/postgres/advanced-guides/high-availability-and-global-replication/) you can create an HA cluster in your primary region and read-only replicas elsewhere. + +And there is no lock in here. We have a list of [recommended external providers](/docs/postgres/getting-started/what-you-should-know/#recommended-external-providers), but you're free to host your database literally anywhere. + +Before moving on, one last observation on the relational database. While you want and need your application to be on the internet, you are much better off if your relational database is NOT directly exposed to the internet, but can only be accessed via your application. That’s the value of an [internal private network](/docs/networking/private-networking/). The private connection from the demo app to the Postgres app was configured automatically for you. diff --git a/deep-dive/rails.html.markerb b/deep-dive/rails.html.markerb new file mode 100644 index 0000000000..a46128fe2d --- /dev/null +++ b/deep-dive/rails.html.markerb @@ -0,0 +1,42 @@ +--- +title: Rails demo reference +layout: docs +nav: demo +--- + +The Rails source is on [GitHub](https://github.com/fly-apps/rails-dictaphone/). +`fly launch` will provide a [`Dockerfile`](https://docs.docker.com/reference/dockerfile/) + and a [`fly.toml`](/docs/reference/configuration/) config file. `fly launch` will + also work with Rails-provided Dockerfiles. When you make changes to your application, you can run + `bin/rails generate dockerfile` to generate updated Dockerfiles. + +How the pieces are put together: + +* The original web-dictaphone app distributed throughout the application: + * [`app/javascript/controllers/main_controls_controller`](https://github.com/fly-apps/rails-dictaphone/blob/main/app/javascript/controllers/main_controls_controller.js) + and [`app/javascript/controllers/sound_clip_controller`](https://github.com/fly-apps/rails-dictaphone/blob/main/app/javascript/controllers/sound_clip_controller.js) + contain the JavaScript, implemented as [Stimulus](https://stimulus.hotwired.dev/handbook/origin) controllers. + * [`app/assets/stylesheets/dictaphone.css`](https://github.com/fly-apps/rails-dictaphone/blob/main/app/assets/stylesheets/dictaphone.css) contains + the stylesheets. + * [`app/views/clips/index.html.erb`](https://github.com/fly-apps/rails-dictaphone/blob/main/app/views/clips/index.html.erb) contains the HTML +* [Active Record](https://guides.rubyonrails.org/active_record_basics.html) will run using Sqlite3 as the database in development mode, and +PostgreSQL in production. + Access to the database is through the `DATABASE_URL` secret. +* [Active Storage](https://guides.rubyonrails.org/active_storage_overview.html) is configured to write to the filesystem + in development and Tigris in production. The following secrets are used to + establish the connection: `AWS_ACCESS_KEY_ID`, `AWS_ENDPOINT_URL_S3`, `AWS_REGION`, `AWS_SECRET_ACCESS_KEY`, and `BUCKET_NAME`. +* [Action Cable](https://guides.rubyonrails.org/action_cable_overview.html) is configured to use an Async adapter + in development and the Redis adapter in production. + +Key points of logic: + +* [`app/controllers/clips_controller.rb`](https://github.com/fly-apps/rails-dictaphone/blob/main/app/controllers/clips_controller.rb) contains + the logic to build responses to requests for the index page, and to GET, PUT, + and DELETE audio clips. +* The realtime implementation code is primarily in [ActionCable](https://guides.rubyonrails.org/action_cable_overview.html), so all that is needed is one line in [`app/models/clip.rb`](https://github.com/fly-apps/rails-dictaphone/blob/6bdf4f639640c9fb55530546dbbed682b65a7df9/app/models/clip.rb#L2) +and one line in [`app/views/layouts/application.html.erb`](https://github.com/fly-apps/rails-dictaphone/blob/6bdf4f639640c9fb55530546dbbed682b65a7df9/app/views/layouts/application.html.erb#L9) +* When the `WHISPER_URL` secret is set, `PUT` requests will cause the audio clips + to be passed to the Whisper server, and responses will be used to update the + PostgreSQL database. This is done using [Active Job](https://guides.rubyonrails.org/active_job_basics.html) + and [Sidekiq](https://sidekiq.org/). + The code for this is in [`app/jobs/whisper_transcribe_job.rb`](https://github.com/fly-apps/rails-dictaphone/blob/main/app/jobs/whisper_transcribe_job.rb). \ No newline at end of file diff --git a/deep-dive/recap.html.markerb b/deep-dive/recap.html.markerb new file mode 100644 index 0000000000..702c69a0b9 --- /dev/null +++ b/deep-dive/recap.html.markerb @@ -0,0 +1,25 @@ +--- +title: Recap +layout: docs +nav: demo +toc: false +order: 9 +--- + +We promised that you could get through this in under an hour. You successfully deployed a full-stack app with both object storage and realtime requirements, and then went on to add AI functionality. + +Up front, we set a goal: **Help you find out if Fly.io is the right place for you.** + +Fly.io can support your apps now, with flexible built-in features, and scale with you later as you grow, with low-level control of Machines and powerful compute in 35 [regions](/docs/reference/regions/). + +You're never locked in; you can always eject entirely and move your entire application elsewhere if things don’t work out. + +How did we do? Visit [community](https://community.fly.io/) and let us know. + +## Learn more + + * [Going to production](/docs/apps/going-to-production/) + * [Support](/docs/about/support/) + * [Pricing](/docs/about/pricing/) + * [Application Monitoring by Sentry](../../reference/sentry/) + * [Application Security by Arcjet](../../reference/arcjet/) diff --git a/deep-dive/redis.html.markerb b/deep-dive/redis.html.markerb new file mode 100644 index 0000000000..327558fe9d --- /dev/null +++ b/deep-dive/redis.html.markerb @@ -0,0 +1,21 @@ +--- +title: Upstash Redis +layout: docs +nav: demo +order: 7 +--- + +
+ Illustration by Annie Ruygt of a puzzle box figure holding two slices of cake +
+ +[Upstash Redis](/docs/upstash/redis/) is used by the deep dive demo app for its [pubsub](https://redis.io/docs/latest/commands/?group=pubsub+external) capabilities, but it can do [so much more](https://upstash.com/docs/redis/overall/rediscompatibility+external). + +In particular, Redis is useful for caching: + +- [Node and Redis](https://redis.io/learn/develop/node/nodecrashcourse/caching+external) +- [Rails and Redis](https://guides.rubyonrails.org/caching_with_rails.html#activesupport-cache-rediscachestore+external) + +In the deep dive demo app, updates are broadcast to all Machines via Redis, and then each Machine informs browser clients +of the update via [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API+external). The client requests +updated information from the application using HTTP GET. diff --git a/deep-dive/tigris.html.markerb b/deep-dive/tigris.html.markerb new file mode 100644 index 0000000000..b34e311605 --- /dev/null +++ b/deep-dive/tigris.html.markerb @@ -0,0 +1,15 @@ +--- +title: Tigris +layout: docs +nav: demo +order: 6 +--- +
+ Illustration by Annie Ruygt Bernard of a blue tiger reading a map with delight +
+ +[Tigris](https://fly.io/docs/tigris/) is a globally distributed object storage service. It essentially requires no configuration, and seamlessly handles multi-regions. If you upload an audio file to a host in Virginia, you can access it from Amsterdam. Even better: subsequent accesses from regions other than the primary region are served locally. + +Unlike relational databases, there may be reasons why you want to make your object store available via the internet. In this demo, the object store starts out private, but you can make it [public](/docs/tigris/#public-buckets) if you want. + +And if you happen to have an existing S3 object store, check out [shadow buckets](/docs/tigris/#migrating-to-tigris-with-shadow-buckets), which enable you to incrementally migrate your data. diff --git a/deep-dive/whisper.html.markerb b/deep-dive/whisper.html.markerb new file mode 100644 index 0000000000..a78be7f09f --- /dev/null +++ b/deep-dive/whisper.html.markerb @@ -0,0 +1,31 @@ +--- +title: Whisper +layout: docs +nav: demo +order: 8 +--- + +Since you are now officially a Fly.io expert, let's dive right in and run the following two commands in the project directory of your deep dive demo app: + +``` +fly launch --attach --from https://github.com/rubys/cog-whisper +fly deploy +``` + +Next try capturing a new audio clip. It will be transcribed automatically by OpenAI Whisper, an automatic speech recognition (ASR) program that converts speech into text. + +## What just happened + +You provisioned a new Machine, this time with an [L40S](https://www.nvidia.com/en-us/data-center/l40s/+external) GPU. +It will stop when not in use. It will restart when a new request comes in. +It is only available on the private network using [Flycast](/docs/networking/flycast/). + +It runs [OpenAI Whisper](https://openai.com/index/whisper/+external) accessed via a [COG](https://github.com/replicate/cog+external) interface. + +This process involves taking audio clips from Tigris, passing them to Whisper, and updating Postgres with the results. +The Node code for this is about [two dozen lines of code](https://github.com/fly-apps/node-dictaphone/blob/1e84a4dece6888dfc68880d146b46511d47391b3/app.js#L102-L129), +and for Rails is about a [dozen](https://github.com/fly-apps/rails-dictaphone/blob/6bdf4f639640c9fb55530546dbbed682b65a7df9/app/jobs/whisper_transcribe_job.rb#L5-L16). + +And, as always, there's no lock in. You can opt to replace this with a machine hosted by [Replicate](https://replicate.com/) or elsewhere. + +**Next:** [Recap](/docs/deep-dive/recap) the deep dive diff --git a/django/advanced-guides.html.md b/django/advanced-guides.html.md new file mode 100644 index 0000000000..414de38d82 --- /dev/null +++ b/django/advanced-guides.html.md @@ -0,0 +1,6 @@ +--- +title: Advanced guides +layout: framework_docs_overview +toc: false +order: 2 +--- diff --git a/django/advanced-guides/staging-environments-with-github-actions.html.md b/django/advanced-guides/staging-environments-with-github-actions.html.md new file mode 100644 index 0000000000..9b3564c866 --- /dev/null +++ b/django/advanced-guides/staging-environments-with-github-actions.html.md @@ -0,0 +1,336 @@ +--- +title: Staging environments with GitHub actions +layout: framework_docs +order: 1 +objective: How to create staging environments on the Fly.io with GitHub actions. +related_pages: + - /docs/django/getting-started/ + - /docs/flyctl/ + - /docs/postgres/ + - /docs/reference/configuration/ +--- + +Creating staging environments for testing changes to our apps can be a challenge. This +guide shows how to use GitHub actions to smoothly create a separate staging +environment for each pull request using the +[`fly-pr-review-apps`](https://github.com/marketplace/actions/pr-review-apps-on-fly-io+external) +action, which will create and deploy our Django project with changes from the specific +pull request. It will also destroy it when it's no longer needed, such as after closing +or merging a pull request. The entire staging process enclosed in one GitHub action, so +we don't have to worry about anything else (**NoOps**). + +## Basic flow + +[GitHub action](https://github.com/features/actions+external) workflows are defined by YAML files +in the `.github/workflows/` directory of our repository. Let's add a new flow that will +create and deploy staging environment for each pull request. But first we need to create +a new repository secret called `FLY_API_TOKEN` to use for authentication. Go to a GitHub +repository page and open: + + _Settings (tab) → Security → Secrets and variables → Actions → Repository secrets → New repository secret_ + +next, create a new secret called `FLY_API_TOKEN` with a value from: + +```cmd +fly tokens org +``` + +
+It's also possible to create a token from the organization dashboard, under the "Tokens" tab. +
+ +Now, we're ready to add a new flow. This is how we can define it in the +`.github/workflows/fly_pr_preview.yml` file: + +```yaml +# .github/workflows/fly_pr_preview.yml + +name: Start preview app + +on: + pull_request: + types: [labeled, synchronize, opened, reopened, closed] + +concurrency: + group: ${{ github.workflow }}-pr-${{ github.event.number }} + cancel-in-progress: true + +permissions: + contents: read + +env: + FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} + +jobs: + preview-app: + if: contains(github.event.pull_request.labels.*.name, 'PR preview app') + runs-on: ubuntu-latest + name: Preview app + environment: + name: pr-${{ github.event.number }} + url: ${{ steps.deploy.outputs.url }} + steps: + - name: Checkout + uses: actions/checkout@v4 + - name: Deploy preview app + uses: superfly/fly-pr-review-apps@1.2.0 + id: deploy + with: + region: waw + org: personal +``` + +It's time to split our configuration up into its component parts: + +- `on → pull_request → types`: specifies [events](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request+external) + on which a new staging project will be deployed: + +```yaml +pull_request: + types: [labeled, synchronize, opened, reopened, closed] +``` + +- `concurrency`: prevents concurrent deploys for the same PR (Pull Request). The `group` + name contains a PR number and workflow name to create a separate group for each + workflow and PR: + +```yaml +concurrency: + group: ${{ github.workflow }}-pr-${{ github.event.number }} + cancel-in-progress: true +``` + +- `env`: makes the `FLY_API_TOKEN` secret available to use for authentication: + +```yaml +env: + FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} +``` + +- `jobs → preview-app → if`: skips deploy on PRs without the + _PR preview app_ + label. Both for safety reasons and to avoid creating a staging environments when no + needed: + +```yaml +if: contains(github.event.pull_request.labels.*.name, 'PR preview app') +``` + +- `jobs → preview-app → environment`: describes the deployment target to show up in a + pull request UI. `steps.deploy.outputs.url` is filled by the + `superfly/fly-pr-review-apps` and will contain the URL of a deployed staging project: + +```yaml +environment: + name: pr-${{ github.event.number }} + url: ${{ steps.deploy.outputs.url }} +``` + +- `jobs → preview-app → steps → with`: specifies all inputs that we want to pass to our + flow. You can check available options in the + [README](https://github.com/superfly/fly-pr-review-apps#inputs+external). We pass the Fly.io + [region](https://fly.io/docs/reference/regions/#fly-io-regions) and organization as a + starting point: + +```yaml +with: + region: waw + org: personal +``` + +The action configured in this way will deploy an app with the name created according to +the following pattern: + +
+`pr-{{ PR number }}-{{ repository owner }}-{{ repository name }}` +
+to the: `https://{{ app name }}.fly.dev`, e.g. + +
+`https://pr-9-felixxm-fly-pr-preview-example.fly.dev/` +
+ +(for PR number 9 in the repository called `pr-preview-example`). + +## Using Postgres cluster + +Using a dedicated staging database is a good practice for test environments. This gives +us more control and appropriate separation from the production environment which +eliminates a potential data leak vector. + +If you don't have a [Postgres cluster](https://fly.io/docs/postgres/) specifically for +testing purposes you can create one with [`fly postgres create`](https://fly.io/docs/flyctl/postgres-create/): + +```cmd +fly postgres create --name pg-fly-pr-staging-preview +``` + +Once created, we can specify it in our action (`jobs → preview-app → steps → with`) +using the `postgres` input: + +```yaml +# .github/workflows/fly_pr_preview.yml + +name: Start preview app + +on: + pull_request: + types: [labeled, synchronize, opened, reopened, closed] + +concurrency: + group: ${{ github.workflow }}-pr-${{ github.event.number }} + cancel-in-progress: true + +permissions: + contents: read + +env: + FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} + +jobs: + preview-app: + if: contains(github.event.pull_request.labels.*.name, 'PR preview app') + runs-on: ubuntu-latest + name: Preview app + environment: + name: pr-${{ github.event.number }} + url: ${{ steps.deploy.outputs.url }} + steps: + - name: Checkout + uses: actions/checkout@v4 + - name: Deploy preview app + uses: superfly/fly-pr-review-apps@1.2.0 + id: deploy + with: + postgres: pg-fly-pr-staging-preview # ← Added + region: waw + org: personal +``` + +With that in place, our staging Postgres cluster will be automatically attached to the +test app, which will make a `DATABASE_URL` environment variable available in the test +VM. + +## Additional release steps + +For performing additional release steps when deploying our staging environment, we can +use a custom [Fly.io TOML configuration file](https://fly.io/docs/reference/configuration/) +dedicated for staging with a release script +[to run before a deployment](https://fly.io/docs/reference/configuration/#run-one-off-commands-before-releasing-a-deployment). +First, make a copy of an existing configuration: + +```cmd +mkdir staging +``` +```cmd +cp fly.toml staging/fly_staging.toml +``` + +Next, create a script (`staging/post_deploy.sh`) to prepare our staging database: + +```bash +# staging/post_deploy.sh +#!/usr/bin/env bash + +# Migrate database. +python /code/manage.py migrate +# Load fixtures with test data. +python /code/manage.py loaddata /code/staging/test_groups.json +``` + +Finally, we need to add `release_command` to the TOML configuration calling our script: + +```toml +# staging/fly_staging.toml + +console_command = "/code/manage.py shell" + +[build] + +[env] +  PORT = "8000" + +[deploy] +  release_command = "sh ./staging/post_deploy.sh" # ← Added. + +... +``` + +and specify the custom TOML configuration in our action +(`jobs → preview-app → steps → with`) using the `config` input: + +```yaml +# .github/workflows/fly_pr_preview.yml + +name: Start preview app + +on: + pull_request: + types: [labeled, synchronize, opened, reopened, closed] + +concurrency: + group: ${{ github.workflow }}-pr-${{ github.event.number }} + cancel-in-progress: true + +permissions: + contents: read + +env: + FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} + +jobs: + preview-app: + if: contains(github.event.pull_request.labels.*.name, 'PR preview app') + runs-on: ubuntu-latest + name: Preview app + environment: + name: pr-${{ github.event.number }} + url: ${{ steps.deploy.outputs.url }} + steps: + - name: Checkout + uses: actions/checkout@v4 + - name: Deploy preview app + uses: superfly/fly-pr-review-apps@1.2.0 + id: deploy + with: + config: staging/fly_staging.toml # ← Added + postgres: pg-fly-pr-staging-preview + region: waw + org: personal +``` + +With this small effort we have staging database created on Fly.io! + +🚨 Be aware that `release_command` is run on a temporary VM and cannot modify the +local storage or state. It's fine to run database operations but not to perform release +steps that attempt to modify a local storage, e.g. collecting static files. Such steps +should be added to the `Dockerfile`. For example, if we want to collect static files, +then add the `collectstatic` management command to the `Dockerfile`: + +```docker +ARG PYTHON_VERSION=3.10-slim-bullseye + +FROM python:${PYTHON_VERSION} + +ENV PYTHONDONTWRITEBYTECODE 1 +ENV PYTHONUNBUFFERED 1 + +RUN mkdir -p /code + +WORKDIR /code + +COPY requirements.txt /tmp/requirements.txt +RUN set -ex && \ + pip install --upgrade pip && \ + pip install -r /tmp/requirements.txt && \ + rm -rf /root/.cache/ +COPY . /code + +# ↓ Added ↓ +RUN set -ex && \ + python /code/manage.py collectstatic --noinput + +EXPOSE 8000 + +CMD ["gunicorn", "--bind", ":8000", "--workers", "2", "hello_pr_preview_example.wsgi"] +``` diff --git a/django/getting-started/existing.html.md b/django/getting-started/existing.html.md index e3c1ebd1c4..7c01fa3929 100644 --- a/django/getting-started/existing.html.md +++ b/django/getting-started/existing.html.md @@ -25,7 +25,7 @@ Be sure to have generated an up-to-date `requirements.txt` file for any new pack ## flyctl -Fly.io has its own command-line utility for managing apps, [flyctl](https://fly.io/docs/hands-on/install-flyctl/). If not already installed, follow the instructions on the [installation guide](https://fly.io/docs/hands-on/install-flyctl/) and [log in to Fly](https://fly.io/docs/getting-started/log-in-to-fly/). +Fly.io has its own command-line utility for managing apps, [flyctl](/docs/flyctl/). If not already installed, follow the instructions on the [installation guide](/docs/flyctl/install/) and [log in to Fly](/docs/getting-started/sign-up-sign-in/). ## Provision Django and Postgres Servers @@ -165,7 +165,7 @@ fly ssh console --pty -C 'python /code/manage.py createsuperuser' ### Secrets -Secrets allow sensitive values, such as credentials and API keys, to be securely passed to your Django applications. You can set, remove, or list all secrets with the [fly secrets](https://fly.io/docs/reference/secrets/) command. +Secrets allow sensitive values, such as credentials and API keys, to be securely passed to your Django applications. You can set, remove, or list all secrets with the [fly secrets](/docs/apps/secrets/) command. ### Git diff --git a/django/getting-started/index.html.md b/django/getting-started/index.html.md index 2146b26651..fb0ef28332 100644 --- a/django/getting-started/index.html.md +++ b/django/getting-started/index.html.md @@ -140,7 +140,7 @@ By default, Django is configured for local development. The [How to Deploy Djang However, for demonstration purposes, we can take some shortcuts. First, in the [`hello_django/settings.py`](https://github.com/fly-apps/hello-django/blob/main/hello_django/settings.py) file update the `ALLOWED_HOSTS` configuration to accept -a host on which it's deployed. Use the [`FLY_APP_NAME`](https://fly.io/docs/reference/runtime-environment/#fly_app_name) +a host on which it's deployed. Use the [`FLY_APP_NAME`](https://fly.io/docs/machines/runtime-environment/#fly_app_name) environment variable for that: ```python @@ -167,7 +167,7 @@ That's it! We're ready to deploy on Fly.io. ## flyctl -Fly.io has its own command-line utility for managing apps, [flyctl](https://fly.io/docs/hands-on/install-flyctl/). If not already installed, follow the instructions on the [installation guide](https://fly.io/docs/hands-on/install-flyctl/) and [log in to Fly.io](https://fly.io/docs/getting-started/log-in-to-fly/). +Fly.io has its own command-line utility for managing apps, [flyctl](/docs/flyctl/). If not already installed, follow the instructions on the [installation guide](/docs/flyctl/install/) and [log in to Fly.io](/docs/getting-started/sign-up-sign-in/). ## Configure and Deploy your Fly App diff --git a/django/index.html.markerb b/django/index.html.markerb index 4d3aedffdb..8deb7b4fec 100644 --- a/django/index.html.markerb +++ b/django/index.html.markerb @@ -20,3 +20,5 @@ Run through the [Starter Django App](./getting-started/) guide to get a feel for ## [Existing Django Apps](./getting-started/existing/) The [Existing Django Apps](./getting-started/existing/) guide walks through deploying a production-ready project to Fly.io. If you are coming from another hosting platform like Heroku there are additional tips for managing the migration. + +## [Advanced Guides](./advanced-guides) diff --git a/elixir/advanced-guides/clustering-from-home-to-your-app-in-fly.html.md b/elixir/advanced-guides/clustering-from-home-to-your-app-in-fly.html.md new file mode 100644 index 0000000000..ab1a5562a7 --- /dev/null +++ b/elixir/advanced-guides/clustering-from-home-to-your-app-in-fly.html.md @@ -0,0 +1,174 @@ +--- +title: Easy Clustering from Home to Fly.io +layout: framework_docs +order: 8 +objective: Guide for connecting a locally running Elixir application to another application running on Fly. +author: mark +status: stable +categories: + - elixir +references: + - /elixir/the-basics/clustering +date: 2024-03-18 +--- + +This explains how to cluster a locally running Elixir application with another Elixir application running on Fly.io. Additionally, a [**bash script** (named `cluster_with_remote`)](https://gist.github.com/brainlid/9e02e95f7d9c65a23312a4df95094d2a) is provided to automate the process of starting the local node and clustering it with the server. + +Here we cover _why_ we might want to do this, _what_ is required to make it work, and _how_ to make it happen. + +## Why cluster a local application with the one on the server? + +Besides being really cool that we can do this, there are some practical reasons as well. + +### AI/ML development + +With a local Elixir application clustered to an application on Fly.io with a GPU attached, we can keep our local development workflow without having a large GPU in our development machine. + +When we leverage [Nx](https://github.com/elixir-nx/nx) and [Bumblebee](https://github.com/elixir-nx/bumblebee), we can easily have the clustered application do all the GPU accelerated AI work and return the processing results seamlessly to our local application. + +It really does feel like the GPU is local when we work this way. + +### Develop and debug a distributed application + +Building a globally distributed application can be challenging to model locally. With [Fly.io Regions](https://fly.io/docs/reference/regions/), we can deploy our cluster-aware application where it makes sense. Then, our local application joins the global cluster, giving us a close-up view of how the application behaves in a truly globally distributed environment. + +## How it works + +Elixir supports [clustering](https://fly.io/docs/elixir/the-basics/clustering/) multiple running Elixir applications together (thanks to [Erlang](https://www.erlang.org/doc/reference_manual/distributed.html)), even while running on separate machines. + +Fly.io makes creating a [WireGuard VPN tunnel](https://fly.io/docs/networking/private-networking/#private-network-vpn) between your local machine and your Fly.io organization easy. + +![Image showing an app inside a house connecting to a wire that plugs into a Fly balloon with a region abbreviation on it.](/docs/images/cluster-from-home-to-fly-app-1.png?center) + +This means we can connect and cluster our locally running Elixir application with an Elixir application deployed at Fly.io. There are some special configuration requirements needed to make this possible, but we cover it all here. + +### Prerequisites + +There are a few prerequisites to consider when clustering a local Elixir application to one running in Fly.io. We'll provide tools or instructions to help with each. + +1. A [WireGuard connection](https://fly.io/docs/networking/private-networking/#private-network-vpn) must be setup and open from your local machine to Fly.io +1. The same version of Elixir and Erlang OTP must be running on both ends. +1. The locally running Elixir application needs the Erlang COOKIE used on the server. +1. We need the full node name of the Elixir node running on the server (**NOTE:** The bash script provided later does this for you.) +1. The local Elixir application must be started with IPv6 networking support enabled. By default it does not. + +### Setting the Erlang cookie + +The [recommended way for setting the Erlang COOKIE](https://fly.io/docs/elixir/the-basics/clustering/#making-the-cookie-changes) value in your deployed application is to set an ENV named `RELEASE_COOKIE`. This gives the server a stable, predictable cookie value. In order for the nodes to connect, they need the same cookie. + +When [set in the ENV through Fly.io](https://fly.io/docs/flyctl/config-env/), we can read it from the application's information. The script does this so we don't have to do anything else. + +### Starting with IPv6 support + +Fly.io uses an IPv6 network internally for private IPs. The BEAM needs IPv6 support to be enabled explicitly. On the server, that’s taken care of through a Dockerfile. Locally, however, it needs to be enabled so the local application can cluster with the remote node. + +The issue is, if IPv6 support is enabled globally, like in a `.bashrc` file, then setting it in the `cluster_with_remote` script essentially flips it OFF. If NOT set globally, then it should be set in the script. Choose the approach that best fits your situation. + +When set globally in your `.bashrc` file (recommended), it looks like this: + +``` +export ERL_AFLAGS="-kernel shell_history enabled -proto_dist inet6_tcp" +``` + +This one includes the added benefit of turning on shell history in IEx. Yay! + +If not set globally, it can be set in the script command like this: + +``` +iex --erl "-proto_dist inet6_tcp" --sname local [...] +``` + +Where the `--erl "-proto_dist inet6_tcp"` portion is the key. + +### Hidden by default: A note about production systems + +Joining your local application to a production cluster in a public way may result in other application sending work, tasks, or executing processes in your local node, depending on the behavior of your specific application. This may not be what you want! + +For this reason, the script uses the `--hidden` option to hide the local node from the rest of the cluster. + +To see the **_all_** the connected nodes, including any hidden ones, use: + +```elixir +Node.list(:hidden) +``` + +
+If you want the local Elixir application to be _fully visible_ to the rest of the cluster, remove the `--hidden` argument in the bash script. +
+ +## Bash script file + +Create [this file](https://gist.github.com/brainlid/9e02e95f7d9c65a23312a4df95094d2a) locally in your root of your Elixir project. + + + +
+ +
+ +Make the script file executable. + +``` +chmod +x cluster_with_remote +``` + +## Script usage + +With the perquisites out of the way, we'll use the `cluster_with_remote` script to start our local Elixir application. The script automates much of what needs to be done to make the process work smoothly. Feel free to customize the script as needed. + +The script is designed to be copied into a project with little to no modification required. The only expected customizations are handled through ENV values that can be controlled per-project. + +There are two primary ways to use the script: + +1. Cluster the local application to the same project deployed at Fly.io. This is the same application running in two places. +1. Cluster a local application to a difference project deployed at Fly.io. The deployed application's name must be provided. + +### Cluster to the same application running on the server + +The simplest variation is when we cluster our local application to a deployed version of itself. The `fly.toml` file in the directory with a copy of the `cluster_with_remote` is for the deployed application. + +To do this, just execute the script: + +``` +./cluster_with_remote +``` + +The script outputs if the clustering connection succeeded and prints the names of the connected nodes. + +### Cluster to a different application running on the server + +There are times when the local application is connecting to a _different_ application than where the script is running from. + +To do this, we need to tell the script the Fly.io app name of the application we want to connect with. It can be done like this: + +``` +CLUSTER_APP_NAME=server-app-name ./cluster_with_remote +``` + +This can also be set as a project-specific ENV using a tool like [direnv](https://direnv.net/) or [dotenv](https://www.dotenv.org/). The custom app name to cluster with can be written to env file and then running the script just works. + +``` +./cluster_with_remote +``` + +If the Erlang cookie of the deployed application is not set using the recommended `RELEASE_COOKIE` ENV setting, it can still be provided to the script using a local ENV named `RELEASE_COOKIE`. See the script for details. + +![Image showing an app inside a house connecting to a wire that plugs into a Fly balloon with a region abbreviation on it. There are three balloons in the sky and for different regions and they are connected by wires.](/docs/images/cluster-from-home-to-fly-app-2.png?center) + +Now you're _really_ doing distributed Elixir! + +## Summary + +When we couple Elixir's clustering ability with Fly.io's networking, VPN, and API discoverability, we can easily cluster a locally running Elixir application with a deployed Elixir application. This makes it easy to leverage hosted GPUs for developing AI/ML applications or working on distributed applications. + +The ready-to-use script automates much of the process. diff --git a/elixir/advanced-guides/connect-livebook-to-your-app.html.md b/elixir/advanced-guides/connect-livebook-to-your-app.html.md index 76092c5b70..2c27960213 100644 --- a/elixir/advanced-guides/connect-livebook-to-your-app.html.md +++ b/elixir/advanced-guides/connect-livebook-to-your-app.html.md @@ -2,8 +2,9 @@ title: Connecting Livebook to Your App in Production layout: framework_docs order: 6 -objective: Guide shows how to connect a locally running Livebook to your application running on Fly. +objective: Guide for connecting a locally running Livebook to your application running on Fly. redirect_from: /docs/app-guides/elixir-livebook-connection-to-your-app +redirect_from: /blog/connecting-livebook-to-your-production-application/ author: mark categories: - elixir @@ -24,7 +25,7 @@ There are a few requirements for this to work. - **Livebook requires Elixir version 1.14.2 or higher**. Livebook runs locally on your machine, so that part is easy to control. However, the **server** side needs to have the same version of Elixir as well. When Livebook connects to the server, it loads some code into the server environment as well. So your server version of Elixir also needs to be 1.14.2 or higher. I recommend making your local version be the same as the production version to reduce potential problems. - **Known cookie value on your server**. Livebook needs to know the cookie value used on the server. Follow [this guide to give your app a static cookie value](/docs/elixir/the-basics/clustering/#the-cookie-situation). -- **WireGuard setup on your local machine**. Follow the Fly.io [Private Network VPN](/docs/reference/private-networking/#private-network-vpn) guide to walk through that. +- **WireGuard setup on your local machine**. Follow the Fly.io [Private Network VPN](/docs/networking/private-networking/#private-network-vpn) guide to walk through that.