Skip to main content

Migrate a self-hosted database to Cloud

You run Serverpod and PostgreSQL yourself, for example with Docker Compose on a VPS. Now you want to move to Serverpod Cloud. This guide takes your data, your users, and their sessions across.

It happens in two halves. Deploying your project to Cloud comes first, because that is what creates the tables from your migrations. Copying the rows comes second, out of your old database and into those tables.

Before you start​

You need:

  • The Serverpod Cloud CLI set up and authenticated. See Set up the Cloud CLI.
  • Your Serverpod project on your machine, with the same code and migrations that run on your server.
  • Shell access to the server that runs your database.
  • The PostgreSQL client tools (pg_restore and psql) on your machine. Use the same major version as your self-hosted database or newer. Cloud runs PostgreSQL 17, so version 17 works unless your self-hosted database is newer. See PostgreSQL downloads.

The commands below use example names. Your database runs in a Docker Compose service called postgres, and your server runs in a service called server. The database is called my_project. Replace these names with your own.

Run the docker compose commands on your server, and the serverpod cloud commands from your project's <project>_server folder on your own machine.

Deploy your project to Cloud​

Create the project with the database enabled, and deploy it:

serverpod cloud launch

Cloud applies your migrations on deploy. Your tables now exist on Cloud, with no app data in them yet. See Deploy your first app for the full walkthrough.

Your self-hosted database must be on the same migration versions. Check which versions it has:

docker compose exec postgres psql -U postgres -d my_project \
  -c "SELECT module, version FROM serverpod_migrations ORDER BY module;"

You run the same query against Cloud later in this guide. If the versions differ, bring your server up to date and deploy the same migrations to both.

Copy your auth secrets to Cloud​

Cloud generates its own auth secrets for a new project. Your users' passwords and sessions depend on the secrets from your server, so they stop working with the new ones:

  • Signing in with a correct password fails with invalidCredentials.
  • Refreshing a session fails with RefreshTokenInvalidSecretException, and the server deletes that refresh token.

Copy the values from the production section of your server's config/passwords.yaml, or from the matching SERVERPOD_PASSWORD_* environment variables:

serverpod cloud password set emailSecretHashPepper "<value from your server>"
serverpod cloud password set jwtHmacSha512PrivateKey "<value from your server>"
serverpod cloud password set jwtRefreshTokenHashPepper "<value from your server>"

Set every other password your server reads the same way, for example serverSideSessionKeyHashPepper or the client secrets for your sign-in providers. If you accepted them when serverpod cloud launch asked, they're already set. Then deploy, so the server picks up the new values:

serverpod cloud deploy
warning

Copy the secrets before you restore any users. When a session refresh fails on a mismatched secret, the server deletes that refresh token. The user is signed out and has to sign in again, and setting the secrets afterwards doesn't bring the session back.

Stop your self-hosted server​

Stop the server, so no new rows are written after you take the dump. Keep PostgreSQL running:

docker compose stop server

Your API is offline from here until your apps point at Cloud.

Dump your data​

Take a data-only dump. Leave out the data Serverpod keeps about each deployment, such as logs, health checks, and migration history. Cloud wrote its own rows for those tables when it deployed your project:

docker compose exec -T postgres pg_dump -U postgres -d my_project \
  --data-only --format=custom \
  --exclude-table-data='serverpod_migrations*' \
  --exclude-table-data='serverpod_health_*' \
  --exclude-table-data='serverpod_*log*' \
  --exclude-table-data='serverpod_readwrite_test*' \
  --exclude-table-data='serverpod_future_call_claim*' \
  > app-data.dump

The dump uses these options:

  • Dump rows only, with --data-only. Your migrations already created the tables on Cloud. The database user you restore with can't create or change tables anyway. A data-only dump also orders tables by their foreign keys, so each row is restored after the rows it points to.
  • Each pattern ends with *. The * also leaves out each table's ID sequence. Without it, the dump carries your server's sequence values, and the restore resets Cloud's counters for those tables.
  • The excluded tables stay behind. Your server's logs and health checks stay in your old database, so Cloud starts with a clean history. Future call claims are short-lived locks held by a running server, so they aren't needed. The future calls themselves are copied.
  • Everything else is included. That covers your own tables, users, sessions, runtime settings, future calls, and files stored in the database.
tip

The pg_dump command warns about circular foreign keys between serverpod_auth_core_profile and serverpod_auth_core_profile_image, with a hint to use a full dump. Ignore the hint. The warning only matters if some of your users have profile images.

Count the users with profile images:

docker compose exec postgres psql -U postgres -d my_project -At \
  -c 'SELECT count(*) FROM serverpod_auth_core_profile WHERE "imageId" IS NOT NULL;'

If the count is 0, skip to Create a database user.

Dump users with profile images​

The serverpod_auth_core_profile and serverpod_auth_core_profile_image tables point at each other, so neither can be restored first. Cloud doesn't let you turn off foreign key checks during a restore either. Instead, you restore the profiles without their image links and add the links back afterwards.

First, save the links as SQL statements. They name the public schema, because pg_restore leaves the search_path empty on the connection it used:

docker compose exec -T postgres psql -U postgres -d my_project -At \
  -c "SELECT format('UPDATE public.serverpod_auth_core_profile SET \"imageId\" = %L WHERE id = %L;', \"imageId\", id) FROM serverpod_auth_core_profile WHERE \"imageId\" IS NOT NULL;" \
  > profile-images.sql

Next, create a copy of the database and clear the links in the copy. Your original database stays untouched. Copying only works while nothing is connected to my_project, and stopping the server took care of that:

docker compose exec postgres psql -U postgres \
  -c "CREATE DATABASE my_project_export TEMPLATE my_project;"
docker compose exec postgres psql -U postgres -d my_project_export \
  -c 'UPDATE serverpod_auth_core_profile SET "imageId" = NULL;'

Then run the pg_dump command from Dump your data again, with -d my_project_export instead of -d my_project.

Create a database user​

The restore connects as a database user that you create yourself. Print the connection details first:

serverpod cloud db connection

The output ends with a psql command that contains your connection string, in the form postgresql://<host>/<database>?sslmode=require. The commands below use that string with migrator@ added after postgresql://.

Now create that user. The password is shown only once, so save it:

serverpod cloud db user create migrator

The migrator user can read and write rows. It can't create or change tables, disable triggers, or turn off foreign key checks. See Access the database directly for more about database users.

Check that Cloud is on the same migration versions as your server:

psql "postgresql://migrator@<host>/<database>?sslmode=require" \
  -c "SELECT module, version FROM serverpod_migrations ORDER BY module;"

Restore the data​

Copy app-data.dump from your server to your machine. One way is scp, which copies files over SSH. Run it on your machine, with your own user, server address, and path:

scp user@your-server:~/my_project/app-data.dump .

If you created profile-images.sql, copy it the same way.

If your project is on the Growth plan, take a backup snapshot first. See Database backups.

Cloud wrote default runtime settings when it deployed your project. Delete them, so the settings from your server can take their place:

psql "postgresql://migrator@<host>/<database>?sslmode=require" \
  -c "DELETE FROM public.serverpod_runtime_settings;"

Restore the dump into Cloud:

pg_restore \
  --dbname="postgresql://migrator@<host>/<database>?sslmode=require" \
  --data-only --single-transaction --exit-on-error \
  app-data.dump

The --single-transaction and --exit-on-error options make the restore all or nothing. If any row fails, nothing is written. Fix the problem and run the same command again.

If you created profile-images.sql, add the image links back:

psql "postgresql://migrator@<host>/<database>?sslmode=require" \
  -v ON_ERROR_STOP=1 -f profile-images.sql

Your server reads its runtime settings when it starts. Deploy again, so it picks up the ones you restored:

serverpod cloud deploy

Check the result​

Count the rows in your most important tables on Cloud:

psql "postgresql://migrator@<host>/<database>?sslmode=require" \
  -c "SELECT count(*) FROM public.serverpod_auth_core_user;"

Run the same query on your server, and compare the numbers. Then call your Cloud API, for example from a debug build of your Flutter app:

  • Call an endpoint that reads your data.
  • Create a new row, and check that it gets the next ID after your migrated rows.
  • Sign in with an existing account.

When everything works, move your apps over to Cloud. Existing sessions keep working, because Cloud now uses your server's auth secrets.

  • Keep your domain. Attach it to your Cloud project, and your apps don't need a new build. See Custom domains.
  • Use your Cloud URL. Your API runs at https://<project-id>.api.serverpod.space/. For mobile and desktop apps, set apiUrl in your Flutter app's assets/config.json to that URL. You can also pass it with --dart-define=SERVER_URL=<url> when you build. Then ship a new build.
  • Flutter web apps deployed with your server get the Cloud URL from the server, so they need no change. This applies to projects created on Serverpod 3.2 or later.

Clean up​

Delete the migrator user:

serverpod cloud db user delete migrator

If you created an export copy, drop it on your server:

docker compose exec postgres psql -U postgres -c "DROP DATABASE my_project_export;"

Keep your self-hosted server and its data until your apps run against Cloud without problems. Then shut it down.

Troubleshooting​

duplicate key value violates unique constraint "serverpod_migrations_pkey". The dump includes data that Cloud already wrote when it deployed your project. The same error can name serverpod_health_metric or serverpod_session_log. Dump again with every --exclude-table-data option from Dump your data. If the error names serverpod_runtime_settings, delete Cloud's runtime settings as shown in Restore the data. With --single-transaction, nothing was written, so you can restore again right away.

violates foreign key constraint. If the constraint is serverpod_auth_core_profile_fk_1, some of your users have profile images. Follow Dump users with profile images. For any other constraint, check that you dumped with --data-only.

Signing in fails with invalidCredentials, or refreshing fails with RefreshTokenInvalidSecretException. Cloud uses different auth secrets from your server. Follow Copy your auth secrets to Cloud. Users whose refresh failed before the fix need to sign in again.

relation "..." does not exist in psql right after a restore. The pg_restore command sets search_path to an empty value on its connection. Cloud pools connections, so a later session can get that connection back with the empty value still set. Run SET search_path TO public; or reconnect later. Your deployed server isn't affected.