Skip to content

README.md

Latest commit

 

History

History
890 lines (657 loc) · 32.4 KB

README.md

File metadata and controls

890 lines (657 loc) · 32.4 KB
Raw

Cloud SQL Auth Proxy

CI

Warning

Go versions 1.25.2 and 1.24.8 are NOT compatible with Cloud Auth Proxy.

An update to the Go version 1.25.2 and Go 1.24.8 breaks SAN verificaton. This is because Cloud SQL includes a trailing dot in the DNS name within the certificate's Subject Alternative Name (SAN), which the above Go versions reject as a malformed DNS name.

For more details, please see the related Go issue: crypto/x509: quadratic complexity when checking name constraints .

Important

The Cloud SQL Auth Proxy does not currently support Unix domain socket connections to MySQL 8.4 instances. This is due to a known issue involving the new default caching_sha2_password authentication plugin.

The Cloud SQL Auth Proxy is a utility for ensuring secure connections to your Cloud SQL instances. It provides IAM authorization, allowing you to control who can connect to your instance through IAM permissions, and TLS 1.3 encryption, without having to manage certificates.

See the Connecting Overview page for more information on connecting to a Cloud SQL instance, or the About the Proxy page for details on how the Cloud SQL Proxy works.

The Cloud SQL Auth Proxy has support for:

If you're using Go, Java, Python, or Node.js, consider using the corresponding Cloud SQL connector which does everything the Proxy does, but in process:

For users migrating from v1, see the Migration Guide. The v1 README is still available.

Important

The Proxy does not configure the network between the VM it's running on and the Cloud SQL instance. You MUST ensure the Proxy can reach your Cloud SQL instance, either by deploying it in a VPC that has access to your Private IP instance, or by configuring Public IP.

Installation

Check for the latest version on the releases page and use the following instructions for your OS and CPU architecture.

Note

Starting with version v2.17.1, Windows binaries provided on the releases page are signed with Google LLC certificates.

Linux amd64 
# see Releases for other versions
URL="https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.25.4"

curl "$URL/cloud-sql-proxy.linux.amd64" -o cloud-sql-proxy

chmod +x cloud-sql-proxy
Linux 386 
# see Releases for other versions
URL="https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.25.4"

curl "$URL/cloud-sql-proxy.linux.386" -o cloud-sql-proxy

chmod +x cloud-sql-proxy
Linux arm64 
# see Releases for other versions
URL="https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.25.4"

curl "$URL/cloud-sql-proxy.linux.arm64" -o cloud-sql-proxy

chmod +x cloud-sql-proxy
Linux arm 
# see Releases for other versions
URL="https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.25.4"

curl "$URL/cloud-sql-proxy.linux.arm" -o cloud-sql-proxy

chmod +x cloud-sql-proxy
Mac (Intel) 
# see Releases for other versions
URL="https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.25.4"

curl "$URL/cloud-sql-proxy.darwin.amd64" -o cloud-sql-proxy

chmod +x cloud-sql-proxy
Mac (Apple Silicon) 
# see Releases for other versions
URL="https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.25.4"

curl "$URL/cloud-sql-proxy.darwin.arm64" -o cloud-sql-proxy

chmod +x cloud-sql-proxy
Windows x64 
# see Releases for other versions
curl https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.25.4/cloud-sql-proxy.x64.exe -o cloud-sql-proxy.exe
Windows x86 
# see Releases for other versions
curl https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.25.4/cloud-sql-proxy.x86.exe -o cloud-sql-proxy.exe

Install from Source

To install from source, ensure you have the latest version of Go installed.

Then, simply run:

go install github.com/GoogleCloudPlatform/cloud-sql-proxy/v2@latest

The cloud-sql-proxy will be placed in $GOPATH/bin or $HOME/go/bin.

Usage

The following examples all reference an INSTANCE_CONNECTION_NAME, which takes the form: myproject:myregion:myinstance.

To find your Cloud SQL instance's INSTANCE_CONNECTION_NAME, visit the detail page of your Cloud SQL instance in the console, or use gcloud with:

gcloud sql instances describe <INSTANCE_NAME> --format='value(connectionName)'

Credentials

The Cloud SQL Proxy uses a Cloud IAM principal to authorize connections against a Cloud SQL instance. The Proxy sources the credentials using Application Default Credentials.

Note

Any IAM principal connecting to a Cloud SQL database will need one of the following IAM roles:

  • Cloud SQL Client (preferred)
  • Cloud SQL Editor
  • Cloud SQL Admin

Or one may manually assign the following IAM permissions:

  • cloudsql.instances.connect
  • cloudsql.instances.get

See Roles and Permissions in Cloud SQL for details.

When the Proxy authenticates under the Compute Engine VM's default service account, the VM must have at least the sqlservice.admin API scope (i.e., "https://www.googleapis.com/auth/sqlservice.admin") and the associated project must have the SQL Admin API enabled. The default service account must also have at least writer or editor privileges to any projects of target SQL instances.

The Proxy also supports two flags related to credentials:

  • --token to use an OAuth2 token
  • --credentials-file to use a service account key file

Basic Usage

To start the Proxy, use:

# starts the Proxy listening on localhost with the default database engine port
# For example:
#   MySQL      localhost:3306
#   Postgres   localhost:5432
#   SQL Server localhost:1433
./cloud-sql-proxy <INSTANCE_CONNECTION_NAME>

The Proxy will automatically detect the default database engine's port and start a corresponding listener. Production deployments should use the --port flag to reduce startup time.

The Proxy supports multiple instances:

./cloud-sql-proxy <INSTANCE_CONNECTION_NAME_1> <INSTANCE_CONNECTION_NAME_2>

Configuring Port

To override the port, use the --port flag:

# Starts a listener on localhost:6000
./cloud-sql-proxy --port 6000 <INSTANCE_CONNECTION_NAME>

When specifying multiple instances, the port will increment from the flag value:

# Starts a listener on localhost:6000 for INSTANCE_CONNECTION_1
# and localhost:6001 for INSTANCE_CONNECTION_NAME_2.
./cloud-sql-proxy --port 6000 <INSTANCE_CONNECTION_NAME_1> <INSTANCE_CONNECTION_NAME_2>

To configure ports on a per instance basis, use the port query param:

# Starts a listener on localhost:5000 for the instance called "postgres"
# and starts a listener on localhost:6000 for the instance called "mysql"
./cloud-sql-proxy \
    'myproject:my-region:postgres?port=5000' \
    'myproject:my-region:mysql?port=6000'

Configuring Listening Address

To override the choice of localhost, use the --address flag:

# Starts a listener on all interfaces at port 5432
./cloud-sql-proxy --address 0.0.0.0 <INSTANCE_CONNECTION_NAME>

To override address on a per-instance basis, use the address query param:

# Starts a listener on 0.0.0.0 for "postgres" at port 5432
# and a listener on 10.0.0.1:3306 for "mysql"
./cloud-sql-proxy \
    'myproject:my-region:postgres?address=0.0.0.0' \
    'myproject:my-region:mysql?address=10.0.0.1"

Configuring Private IP

By default, the Proxy attempts to connect to an instance's public IP. To enable private IP, use:

# Starts a listener connected to the private IP of the Cloud SQL instance.
# Note: there must be a network path present for this to work.
./cloud-sql-proxy --private-ip <INSTANCE_CONNECTION_NAME>

Important

The Proxy does not configure the network. You MUST ensure the Proxy can reach your Cloud SQL instance, either by deploying it in a VPC that has access to your Private IP instance, or by configuring Public IP.

Configuring Unix domain sockets

The Proxy also supports Unix domain sockets. To start the Proxy with Unix sockets, run:

# Uses the directory "/mycooldir" to create a Unix socket
# For example, the following directory would be created:
#   /mycooldir/myproject:myregion:myinstance
./cloud-sql-proxy --unix-socket /mycooldir <INSTANCE_CONNECTION_NAME>

To configure a Unix domain socket on a per-instance basis, use the unix-socket query param:

# Starts a TCP listener on localhost:5432 for "postgres"
# and creates a Unix domain socket for "mysql":
#     /cloudsql/myproject:my-region:mysql
./cloud-sql-proxy \
    myproject:my-region:postgres \
    'myproject:my-region:mysql?unix-socket=/cloudsql'

Note

The Proxy supports Unix domain sockets on recent versions of Windows, but replaces colons with periods:

# Starts a Unix domain socket at the path:
#    C:\cloudsql\myproject.my-region.mysql
./cloud-sql-proxy --unix-socket C:\cloudsql myproject:my-region:mysql

Configuring IAM Database Authentication

The Proxy supports Automatic IAM Database Authentication for MySQL and Postgres instances, allowing IAM principal's to authenticate and connect as database users.

Make sure to configure your Cloud SQL instance to allow IAM authentication and to add your IAM principal as a database user.

./cloud-sql-proxy --auto-iam-authn <INSTANCE_CONNECTION_NAME>

Important

Make sure to run the Proxy as the same IAM principal as the database user you want to log in as. Only the IAM principal that is attached to the sourced credentials will be able to successfully log in via automatic IAM database authentication.

When logging in using an IAM database user, Cloud SQL truncates usernames based on the engine type in order to not exceed character limits. PostgreSQL's username character limit is 63, while MySQL's is 32.

Cloud SQL IAM database usernames are formatted in the following way:

Postgres:

  • For an IAM user account, this is the user's email address.
  • For a service account, it is the service account's email without the .gserviceaccount.com domain suffix.

MySQL:

  • For an IAM user account, this is the user's email address, without the @ or domain name. For example, for test-user@gmail.com, the database user would be test-user.
  • For a service account, this is the service account's email address without the @project-id.iam.gserviceaccount.com suffix.

Configuring Service Account Impersonation

The Proxy supports service account impersonation. This allows the Proxy to act as a different service account, which can be useful for granting access to resources that are not accessible to the default IAM principal.

To use service account impersonation, you must have the iam.serviceAccounts.getAccessToken permission on the IAM principal impersonating another service account. You can grant this permission by assigning the roles/iam.serviceAccountTokenCreator role to the IAM principal.

To impersonate a service account, use the --impersonate-service-account flag:

Note

The impersonated service account must have the Service Usage Consumer and Cloud SQL Client permissions. Additionally, to use IAM Authenticated users, add the Cloud SQL Instance User permission.

# Starts a listener on localhost:5432 and impersonates the service account
# "my-other-sa@my-project.iam.gserviceaccount.com".
# The Proxy will use the credentials of the principal running the Proxy to
# generate a short-lived access token for the impersonated service account.
./cloud-sql-proxy --impersonate-service-account \
my-other-sa@my-project.iam.gserviceaccount.com <INSTANCE_CONNECTION_NAME>

Using Advanced Disaster Recovery and DNS domain names to identify instances

The proxy can be configured to use DNS to look up an instance. Use a DNS name managed by Cloud SQL Advanced Disaster Recovery, or a domain name that you manage.

Using Advanced Recovery Write Endpoint DNS Name

Advanced Disaster Recovery creates geographically distributed replicas of your Cloud SQL database instance. When you perform switchover or failover on the database instance, the proxy will gracefully disconnect from the old primary instance and reconnect to the new primary instance.

Follow the instructions in Connect using Write Endpoint to get the write endpoint DNS name for your primary instance. Then, use this write endpoint DNS name to configure the proxy.

Configure your DNS Records

The proxy may be configured to use DNS that you define as well.

Add a DNS TXT record for the Cloud SQL instance to a private DNS server or a private Google Cloud DNS Zone used by your application.

Note: You are strongly discouraged from adding DNS records for your Cloud SQL instances to a public DNS server. This would allow anyone on the internet to discover the Cloud SQL instance name.

For example: suppose you wanted to use the domain name prod-db.mycompany.example.com to connect to your database instance my-project:region:my-instance. You would create the following DNS record:

  • Record type: TXT
  • Name: prod-db.mycompany.example.com – This is the domain name used by the application
  • Value: my-project:region:my-instance – This is the instance name

Configuring the Proxy

Configure the Proxy with your DNS domain name instead of an instance connection name:

./cloud-sql-proxy prod-db.mycompany.example.com

Automatic fail-over using DNS domain names

When the Proxy is configured using a domain name, it will periodically check if the DNS record for an instance changes. When the Proxy detects that the domain name refers to a different instance, it will close all open connections to the old instance. Subsequent connection attempts will be directed to the new instance.

For example: suppose application is configured to connect using the domain name prod-db.mycompany.example.com. Initially the corporate DNS zone has a TXT record with the value my-project:region:my-instance. The application establishes connections to the my-project:region:my-instance Cloud SQL instance.

Then, to reconfigure the application to use a different database instance, change the value of the prod-db.mycompany.example.com DNS record from my-project:region:my-instance to my-project:other-region:my-instance-2

The Proxy detects the change to this DNS record. Now, when the application connects to its database using the domain name prod-db.mycompany.example.com, it will connect to the my-project:other-region:my-instance-2 Cloud SQL instance.

The Proxy will automatically close all existing connections to my-project:region:my-instance. This will force the connection pools to establish new connections. Also, it may cause database queries in progress to fail.

The Proxy will poll for changes to the DNS name every 30 seconds by default.

Testing Connectivity

The Proxy includes support for a connection test on startup. This test helps ensure the Proxy can reach the associated instance and is a quick debugging tool. The test will attempt to connect to the specified instance(s) and fail if the instance is unreachable. If the test fails, the Proxy will exit with a non-zero exit code.

./cloud-sql-proxy --run-connection-test <INSTANCE_CONNECTION_NAME>

Config file

The Proxy supports a configuration file. Supported file types are TOML, JSON, and YAML. Load the file with the --config-file flag:

./cloud-sql-proxy --config-file /path/to/config.[toml|json|yaml]

The configuration file format supports all flags. The key names should match the flag names. For example:

# use instance-connection-name-0, instance-connection-name-1, etc.
# for multiple instances
instance-connection-name = "proj:region:inst"
auto-iam-authn = true
debug = true
debug-logs = true

Run ./cloud-sql-proxy --help for more details. See the full documentation in docs/cmd.

Config environment variables

The proxy supports configuration through environment variables. Each environment variable uses "CSQL_PROXY" as a prefix and is the uppercase version of the flag using underscores as word delimiters.

For example, the --auto-iam-authn flag may be set with the environment variable CSQL_PROXY_AUTO_IAM_AUTHN.

An invocation of the Proxy using environment variables would look like the following:

CSQL_PROXY_AUTO_IAM_AUTHN=true \ 
    ./cloud-sql-proxy <INSTANCE_CONNECTION_NAME>

Run ./cloud-sql-proxy --help for more details.

Configuring a Lazy Refresh

The --lazy-refresh flag configures the Proxy to retrieve connection info lazily and as-needed. Otherwise, no background refresh cycle runs. This setting is useful in environments where the CPU may be throttled outside of a request context, e.g., Cloud Run, Cloud Functions, etc.

Additional flags

To see a full list of flags, use:

./cloud-sql-proxy --help

Container Images

There are containerized versions of the Proxy available from the following Artifact Registry repositories:

  • gcr.io/cloud-sql-connectors/cloud-sql-proxy
  • us.gcr.io/cloud-sql-connectors/cloud-sql-proxy
  • eu.gcr.io/cloud-sql-connectors/cloud-sql-proxy
  • asia.gcr.io/cloud-sql-connectors/cloud-sql-proxy

Note

The above container images were migrated from Google Container Registry (deprecated) to Artifact Registry which is why they begin with the old naming pattern (gcr.io)

Each image is tagged with the associated Proxy version. The following tags are currently supported:

  • $VERSION (default)
  • $VERSION-alpine
  • $VERSION-bookworm

The $VERSION is the Proxy version without the leading "v" (e.g., 2.25.4).

For example, to pull a particular version, use a command like:

# $VERSION is 2.25.4
docker pull gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.25.4

We recommend pinning to a specific version tag and using automation with a CI pipeline to update regularly.

The default container image uses distroless with a non-root user. If you need a shell or related tools, use the Alpine or Debian-based container image (bookworm) listed above.

Working with Docker and the Proxy

The containers have the proxy as an ENTRYPOINT so, to use the proxy from a container, all you need to do is specify options using the command, and expose the proxy's internal port to the host. For example, you can use:

docker run --publish <host-port>:<proxy-port> \
    gcr.io/cloud-sql-connectors/cloud-sql-proxy:latest \
    --address "0.0.0.0" --port <proxy-port> <instance-connection-name>

You'll need the --address "0.0.0.0" so that the proxy doesn't only listen for connections originating from within the container.

You will need to authenticate using one of the methods outlined in the credentials section. If using a credentials file you must mount the file and ensure that the non-root user that runs the proxy has read access to the file. These alternatives might help:

  1. Change the group of your local file and add read permissions to the group with chgrp 65532 key.json && chmod g+r key.json.
  2. If you can't control your file's group, you can directly change the public permissions of your file by doing chmod o+r key.json.

Warning

This can be insecure because it allows any user in the host system to read the credential file which they can use to authenticate to services in GCP.

For example, a full command using a JSON credentials file might look like

docker run \
    --publish <host-port>:<proxy-port> \
    --mount type=bind,source="$(pwd)"/sa.json,target=/config/sa.json \
    gcr.io/cloud-sql-connectors/cloud-sql-proxy:latest \
    --address 0.0.0.0 \
    --port <proxy-port> \
    --credentials-file /config/sa.json <instance-connection-name>

Running as a Kubernetes Sidecar

See the example here as well as Connecting from Google Kubernetes Engine.

Running behind a Socks5 proxy

The Cloud SQL Auth Proxy includes support for sending requests through a SOCKS5 proxy. If a SOCKS5 proxy is running on localhost:8000, the command to start the Cloud SQL Auth Proxy would look like:

ALL_PROXY=socks5://localhost:8000 \
HTTPS_PROXY=socks5://localhost:8000 \
    cloud-sql-proxy <INSTANCE_CONNECTION_NAME>

The ALL_PROXY environment variable specifies the proxy for all TCP traffic to and from a Cloud SQL instance. The ALL_PROXY environment variable supports socks5 and socks5h protocols. To route DNS lookups through a proxy, use the socks5h protocol.

The HTTPS_PROXY (or HTTP_PROXY) specifies the proxy for all HTTP(S) traffic to the SQL Admin API. Specifying HTTPS_PROXY or HTTP_PROXY is only necessary when you want to proxy this traffic. Otherwise, it is optional. See http.ProxyFromEnvironment for possible values.

Support for Metrics and Tracing

The Proxy supports Cloud Monitoring, Cloud Trace, and Prometheus.

Supported metrics include:

  • cloudsqlconn/dial_latency: The distribution of dialer latencies (ms)
  • cloudsqlconn/open_connections: The current number of open Cloud SQL connections
  • cloudsqlconn/dial_failure_count: The number of failed dial attempts
  • cloudsqlconn/refresh_success_count: The number of successful certificate refresh operations
  • cloudsqlconn/refresh_failure_count: The number of failed refresh operations.

Supported traces include:

  • cloud.google.com/go/cloudsqlconn.Dial: The dial operation including refreshing an ephemeral certificate and connecting the instance
  • cloud.google.com/go/cloudsqlconn/internal.InstanceInfo: The call to retrieve instance metadata (e.g., database engine type, IP address, etc)
  • cloud.google.com/go/cloudsqlconn/internal.Connect: The connection attempt using the ephemeral certificate
  • SQL Admin API client operations

To enable Cloud Monitoring and Cloud Trace, use the --telemetry-project flag with the project where you want to view metrics and traces. To configure the metrics prefix used by Cloud Monitoring, use the --telemetry-prefix flag. When enabling telemetry, both Cloud Monitoring and Cloud Trace are enabled. To disable Cloud Monitoring, use --disable-metrics. To disable Cloud Trace, use --disable-traces.

To enable Prometheus, use the --prometheus flag. This will start an HTTP server on localhost with a /metrics endpoint. The Prometheus namespace may optionally be set with --prometheus-namespace.

Debug logging

To enable debug logging to report on internal certificate refresh operations, use the --debug-logs flag. Typical use of the Proxy should not require debug logs, but if you are surprised by the Proxy's behavior, debug logging should provide insight into internal operations and can help when reporting issues.

Localhost Admin Server

The Proxy includes support for an admin server on localhost. By default, the the admin server is not enabled. To enable the server, pass the --debug or --quitquitquit flag. This will start the server on localhost at port 9091. To change the port, use the --admin-port flag.

When --debug is set, the admin server enables Go's profiler available at /debug/pprof/.

See the documentation on pprof for details on how to use the profiler.

When --quitquitquit is set, the admin server adds an endpoint at /quitquitquit. The admin server exits gracefully when it receives a GET or POST request at /quitquitquit.

Frequently Asked Questions

Why would I use the Proxy?

The Proxy is a convenient way to control access to your database using IAM permissions while ensuring a secure connection to your Cloud SQL instance. When using the Proxy, you do not have to manage database client certificates, configured Authorized Networks, or ensure clients connect securely. The Proxy handles all of this for you.

How should I use the Proxy?

The Proxy is a gateway to your Cloud SQL instance. Clients connect to the Proxy over an unencrypted connection and are authorized using the environment's IAM principal. The Proxy then encrypts the connection to your Cloud SQL instance.

Because client connections are not encrypted and authorized using the environment's IAM principal, we recommend running the Proxy on the same VM or Kubernetes pod as your application and using the Proxy's default behavior of allowing connections from only the local network interface. This is the most secure configuration: unencrypted traffic does not leave the VM, and only connections from applications on the VM are allowed.

Here are some common examples of how to run the Proxy in different environments:

Why can't the Proxy connect to my private IP instance?

The Proxy does not configure the network between the VM it's running on and the Cloud SQL instance. You MUST ensure the Proxy can reach your Cloud SQL instance, either by deploying it in a VPC that has access to your Private IP instance, or by configuring Public IP.

Should I use the Proxy for large deployments?

We recommend deploying the Proxy on the host machines that are running the application. However, large deployments may exceed the request quota for the SQL Admin API . If your Proxy reports request quota errors, we recommend deploying the Proxy with a connection pooler like pgbouncer or ProxySQL. For details, see Running the Cloud SQL Proxy as a Service.

Can I share the Proxy across multiple applications?

Instead of using a single Proxy across multiple applications, we recommend using one Proxy instance for every application process. The Proxy uses the context's IAM principal and so have a 1-to-1 mapping between application and IAM principal is best. If multiple applications use the same Proxy instance, then it becomes unclear from an IAM perspective which principal is doing what.

How do I verify the shasum of a downloaded Proxy binary?

After downloading a binary from the releases page, copy the sha256sum value that corresponds with the binary you chose.

Then run this command (make sure to add the asterisk before the file name):

echo '<RELEASE_PAGE_SHA_HERE> *<NAME_OF_FILE_HERE>' | shasum -c

For example, after downloading the v2.1.0 release of the Linux AMD64 Proxy, you would run:

$ echo "547b24faf0dfe5e3d16bbc9f751dfa6b34dfd5e83f618f43a2988283de5208f2 *cloud-sql-proxy" | shasum -c
cloud-sql-proxy: OK

If you see OK, the binary is a verified match.

Reference Documentation

Support policy

Major version lifecycle

This project uses semantic versioning, and uses the following lifecycle regarding support for a major version:

  • Active - Active versions get all new features and security fixes (that wouldn’t otherwise introduce a breaking change). New major versions are guaranteed to be "active" for a minimum of 1 year.

  • Maintenance - Maintenance versions continue to receive security and critical bug fixes, but do not receive new features.

Release cadence

The Cloud SQL Auth Proxy aims for a minimum monthly release cadence. If no new features or fixes have been added, a new PATCH version with the latest dependencies is released.

We support releases for 1 year from the release date.

Contributing

Contributions are welcome. Please, see the CONTRIBUTING document for details.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Contributor Code of Conduct for more information.