This page is for developers, data engineers and cloud or DevOps engineers facing a Google Cloud round, from a first cloud role to a senior platform job. Most GCP interviews open with the resource hierarchy and IAM, then ask you to pick between Compute Engine, GKE, Cloud Run and App Engine. After that come VPC networking, Cloud Storage classes, BigQuery and what makes a query expensive, Pub/Sub delivery, and Dataflow or Dataproc at a glance. Senior rounds add a design, an outage and a bill that jumped overnight. Each question shows what the interviewer is checking, the shape of a strong answer and a short answer you can say out loud.
Search all questions by round, difficulty and level, or save the ones you want to practise.
Levels: organization at the top, then folders, then projects, then the resources inside them.
Projects: the boundary for APIs, billing, quotas and most IAM grants.
Inheritance: IAM and organization policies set on a folder apply to every project under it.
"At the top there's the organization node, tied to the company's domain. Under it you create folders, usually for departments or environments, and inside folders you have projects. Every resource, like a VM or a bucket, lives in exactly one project. The project is what you really work with day to day: it's where APIs get enabled, where billing is attached, where quotas apply, and where most IAM roles get granted. The reason the hierarchy matters is inheritance. If I grant a role on a folder, every project under it gets that grant too, and organization policies, like blocking public IPs on VMs, flow down the same way. So I'd set up folders for prod and non-prod, put guardrails high up, and keep one project per app per environment so a mistake in dev can't touch production."
Treating projects as just folders for tidiness, with no mention of billing, quotas or policy inheritance.
Basic: Owner, Editor and Viewer; very broad, across almost every service in the project.
Predefined: Google-managed roles scoped to one service and job, like a storage object viewer.
Custom: your own list of permissions when no predefined role fits; you maintain it.
"IAM grants roles to principals, and a role is just a bundle of permissions. Basic roles are Owner, Editor and Viewer, and they cover nearly every service in the project, so Editor can change almost anything. I avoid them outside a personal sandbox. Predefined roles are maintained by Google and scoped to one service and one kind of job, like read-only access to Cloud Storage objects or running BigQuery jobs. That's where I start. If a predefined role still gives too much, I create a custom role with only the permissions needed, at the organization or project level. The catch is that I own it: when a service adds permissions, a custom role doesn't pick them up. I also grant at the smallest scope that works, a bucket or dataset rather than the whole project, and prefer groups over individual users."
gcloud projects add-iam-policy-binding my-project \
--member="group:data-readers@example.com" \
--role="roles/bigquery.dataViewer"
Handing out Editor to make an error go away, or not knowing that predefined roles exist.
Attach: give the service its own user-managed service account with only the role it needs.
No keys: client libraries use Application Default Credentials and fetch short-lived tokens from the metadata server.
Outside GCP: use Workload Identity Federation instead of exporting a key.
"I'd create a dedicated service account for that one service, grant it a read role on just that bucket, and deploy the Cloud Run service to run as it. Then the code doesn't handle any secret at all. The client library uses Application Default Credentials, which on Cloud Run means it asks the metadata server for a short-lived access token for that service account. Nothing to rotate, nothing to leak. What I avoid is downloading a JSON key and baking it into the image or an environment variable, because that key is long-lived and ends up in places you can't track. I also avoid running as the default compute service account, since it's often shared and over-privileged. If the caller lived outside Google Cloud, say a CI pipeline or another cloud, I'd use Workload Identity Federation so it swaps its own identity token for a short-lived Google one."
Downloading a service account key and putting it in the container image or source code.
Additive: effective access is the union of allow policies on the resource and every ancestor.
Consequence: a lower-level allow policy can't take away an inherited grant.
Options: narrow the folder grant, move the project, or attach an IAM deny policy.
"No, not with an allow policy. In Google Cloud, allow policies are inherited and additive: what a principal can do on a project is the union of what's granted on the project, its folders and the organization. So removing a binding at the project changes nothing if the grant sits on the folder. I'd first ask whether the folder grant is right at all. Often the cleanest fix is to move that grant down to the specific projects that need it, or to move the sensitive project into its own folder. If the folder grant has to stay, IAM deny policies are built for this: you attach a deny policy that blocks specific permissions for that group, and deny is checked before allow. I'd test it with the policy troubleshooter before relying on it, because deny policies only cover the permissions you list."
Saying you'd simply remove the role in the project's IAM page and expect the access to disappear.
Contain: disable the key right away, then delete it once the app has another way in.
Investigate: audit logs for that key, looking for new VMs, new keys and IAM changes.
Clean up: remove anything created, rotate other secrets it could reach.
Prevent: block key creation by policy and move to keyless auth.
"I treat the key as compromised the moment it was public, even if we delete the commit, because bots scan public repos within minutes. First I disable that key, which stops it working immediately while leaving me a record, and I'd delete it once I'm sure what depends on it. Then I check the audit logs for activity using that key since it was pushed: new VMs, especially in odd regions for crypto mining, new service account keys, IAM policy changes, and reads of buckets or secrets. Anything the attacker created gets removed, and any secrets that account could read get rotated. I'd tell the security team and follow our incident process rather than quietly fixing it. Afterwards, I'd find out why a key existed at all, switch that workload to an attached service account or Workload Identity Federation, and enforce the organization policy that blocks new key creation."
gcloud iam service-accounts keys list \
--iam-account=etl-runner@my-project.iam.gserviceaccount.com
gcloud iam service-accounts keys disable KEY_ID \
--iam-account=etl-runner@my-project.iam.gserviceaccount.com
Only deleting the file from the repository and assuming the key is now safe.
Compute Engine: full VMs; for software that needs the OS, special licences or lift-and-shift.
GKE: managed Kubernetes; for many containerised services and teams who want Kubernetes control.
Cloud Run: serverless containers that scale to zero; the easy default for stateless HTTP work and jobs.
App Engine: older platform-as-a-service for web apps in supported runtimes.
"I think of it as a scale from control to convenience. Compute Engine gives me full virtual machines, so I use it when I need the operating system, a vendor product that expects a VM, GPUs with custom drivers, or a straight lift-and-shift. GKE is managed Kubernetes, which makes sense when there are many containerised services, the team already knows Kubernetes, and we need things like sidecars, stateful sets or fine control over scheduling. Cloud Run runs a container without me managing servers or a cluster. It scales on requests, down to zero, so for a stateless API, a webhook or a batch job it's usually my first choice. App Engine is the older platform for web apps where you push code in a supported runtime. It still runs plenty of apps, but for new container work I'd usually reach for Cloud Run instead."
Choosing GKE for a single small API by default, or not being able to say what Cloud Run is for.
Template: an instance template defines machine type, image, disks and startup script.
Autoscaling: a policy on CPU, load balancer usage or a monitoring metric, with sensible minimums.
Autohealing: an application health check recreates VMs that stop answering.
Regional: spread instances across zones and roll out new templates gradually.
"A managed instance group keeps a set of identical VMs built from one instance template, which holds the machine type, the image and the startup script. Because every VM is disposable, the group can add, remove or recreate them freely. For autoscaling I set a minimum and maximum and a target, usually CPU utilisation or load balancer serving capacity for a web tier, or a Cloud Monitoring metric like queue depth for workers. I'd also set an initialisation period so a VM that's still booting doesn't skew the numbers. Autohealing is separate: I attach a health check that hits a real endpoint on the app, and if a VM keeps failing it gets recreated. For production I use a regional group so the VMs spread across zones, and I roll out new templates with a rolling update so only a few instances change at a time."
Keeping state on the VMs in an autoscaled group, or confusing autoscaling with autohealing.
Model: each instance handles several requests at once; Cloud Run adds instances as load grows and can scale to zero.
Cold start: a new instance has to start the container and the app before serving.
Fixes: minimum instances, a smaller image and faster startup, startup CPU boost, tuned concurrency.
"Cloud Run runs copies of my container, and each instance can serve several requests at the same time, up to the concurrency I set. When requests pile up it starts more instances, and when traffic stops it can go all the way to zero. A cold start is the delay when a request needs a fresh instance: pulling the image, starting the runtime and running my init code. For a latency-sensitive API I'd first set a small number of minimum instances so some are always warm, knowing I pay for them while idle. Then I'd make startup cheap: a slimmer image, lazy-loading heavy clients, and no big downloads at boot. Startup CPU boost helps runtimes that are slow to warm up. I'd also check concurrency, because setting it to one forces a new instance for nearly every burst."
Assuming instances keep local state between requests or never restart.
Trigger: an Eventarc trigger on the object finalized event for that bucket.
Handler: read the bucket and object name from the event, then do the work.
Safety: write output to another bucket, make it idempotent, and log failures clearly.
"Cloud Functions, now offered as Cloud Run functions, can be triggered by a Cloud Storage event. I'd use the object finalized event, which fires when a new object is written. In Python I decorate the handler as a CloudEvent function; the event data carries the bucket and the object name. Here I read the file, process it and write the result to a separate output bucket. That last part matters: if I wrote back into the same bucket, my own output would trigger the function again and loop. Events can be delivered more than once, so the work should be idempotent: writing to a name derived from the input means a repeat just overwrites the same result. If I turn on retries, I make sure a bad file fails in a way I can see in the logs rather than retrying forever."
import functions_framework
from google.cloud import storage
client = storage.Client()
OUTPUT_BUCKET = 'reports-processed'
@functions_framework.cloud_event
def on_upload(cloud_event):
data = cloud_event.data
bucket, name = data['bucket'], data['name']
if not name.endswith('.csv'):
return
text = client.bucket(bucket).blob(name).download_as_text()
rows = len(text.splitlines())
out = client.bucket(OUTPUT_BUCKET).blob(name + '.summary.txt')
out.upload_from_string(f'rows={rows}\n')
print(f'processed gs://{bucket}/{name}: {rows} rows')
Writing the output into the bucket that triggers the function, or assuming each event arrives exactly once.
Restore: send all traffic back to the previous healthy revision.
Diagnose: revision logs for startup failures, port, missing secrets, permissions, memory limits.
Prevent: deploy with no traffic, test on a tagged URL, then shift traffic gradually.
"First I get users back to working. Cloud Run keeps earlier revisions, so I send all traffic to the last healthy one, which takes seconds and needs no rebuild. Then I look at the new revision's logs calmly. The usual suspects after a deploy are: the container crashing on startup, the app not listening on the port Cloud Run passes in, a new environment variable or secret that wasn't set, the service account missing a role for something the new code calls, or the instance running out of memory. The logs and the revision's metrics usually point straight at one of those. Once it's fixed, I'd change how we ship: deploy the new revision with no traffic and a tag, test it on its own URL, then move a small share of traffic, watch errors and latency, and only then go all the way."
gcloud run revisions list --service=orders-api --region=europe-west1
gcloud run services update-traffic orders-api \
--region=europe-west1 \
--to-revisions=orders-api-00041-abc=100
Debugging the broken revision live while users keep getting errors, instead of rolling back first.
Scope: the VPC network is global; each subnet belongs to one region with its own IP range.
Defaults: implied rules deny all incoming traffic and allow all outgoing.
Rules: allow or deny, by direction, priority, source range and target tags or service accounts.
"In Google Cloud a VPC network is a global resource, and subnets are regional. So one network can have a subnet in two regions, and VMs in both can talk over internal IPs without extra setup. Routes and firewall rules belong to the network as a whole. Every network has two implied rules: deny all ingress and allow all egress, so nothing gets in until I allow it. A firewall rule has a direction, allow or deny, a priority where the lower number wins, ports and protocols, and a source like an IP range. The targets can be every instance, instances with a network tag, or instances running as a particular service account. I prefer targeting by service account, since only people allowed to use that account can put a VM behind the rule, while anyone who can edit an instance can add a tag."
Saying a VPC lives in one region, or opening port 22 to the whole internet as the normal way to reach a VM.
Internet egress: Cloud NAT on a Cloud Router, per region, outbound only.
Google APIs: Private Google Access on the subnet, or Private Service Connect endpoints.
Admin access: reach the VMs through IAP TCP forwarding instead of public SSH.
"Keeping VMs without external IPs is a good default, so I'd give them two outbound paths. For the public internet, like package mirrors or a partner API, I'd set up Cloud NAT in each region, attached to a Cloud Router. It lets private VMs start outbound connections, but nothing on the internet can start a connection to them. For Google APIs such as Cloud Storage or BigQuery, I'd turn on Private Google Access on the subnet, so the VMs reach those APIs even without an external address. If the security team wants the API traffic to use a private IP inside our network, Private Service Connect gives an internal endpoint for Google APIs. And for people who need to SSH in, I'd use Identity-Aware Proxy TCP forwarding with a firewall rule allowing only IAP's range, rather than opening port 22 to everyone."
Giving every VM a public IP just so it can reach the internet or Cloud Storage.
Shared VPC: one host project owns the network; service projects put their resources in its subnets.
Peering: two separate VPCs exchange routes privately; each keeps its own admins and firewall.
Limits: peering isn't transitive and ranges can't overlap, so plan IP space early.
"Shared VPC is for when one organization wants central control of networking. A host project owns the VPC, subnets, firewall rules and connections back to the office, and each team's service project deploys VMs or clusters into subnets they've been allowed to use. The network team runs the network; the app teams run their apps. VPC Network Peering connects two separate VPCs so they can talk over internal IPs, and each side keeps its own administration. It works across projects and even across organizations, so it suits a partner or a separately owned platform. The traps with peering are that it's not transitive, so if A peers with B and B with C, A can't reach C through B, and the IP ranges can't overlap. For many teams in one company, I'd go Shared VPC, and plan the IP ranges before anyone builds anything."
Building a mesh of peerings between every team's network and expecting traffic to pass through one into another.
Classes: Standard, Nearline, Coldline and Archive, from frequently read to almost never read.
Trade: colder classes cost less to store but add retrieval charges and a minimum storage duration.
Automation: lifecycle rules change class or delete by age, or Autoclass manages it for you.
"Cloud Storage has four main classes. Standard is for data read often, like website assets or active datasets. Nearline suits data read about once a month, Coldline about once a quarter, and Archive is for data you might read once a year, like compliance backups. All of them use the same API and give fast access; what changes is the price shape. The colder the class, the cheaper it is to store but the more it costs to read, and there's a minimum storage duration: thirty days for Nearline, ninety for Coldline and a year for Archive, so deleting early still charges the rest. To automate it, I'd add Object Lifecycle Management rules, for example move objects to Coldline after ninety days and delete them after a few years. If access patterns are unpredictable, I'd turn on Autoclass and let it move objects based on actual use."
Thinking colder classes are slow to read like tape, or ignoring retrieval charges and minimum durations.
Access model: uniform bucket-level access, IAM only, least privilege at the bucket.
Guardrails: public access prevention, enforced by organization policy.
Protection: encryption with customer-managed keys if required, versioning or retention, Data Access audit logs.
Sharing: signed URLs for short-lived access instead of public objects.
"I'd start with uniform bucket-level access, so permissions come only from IAM and there are no per-object ACLs hiding somewhere. Then I'd turn on public access prevention, and enforce both through organization policy so a new bucket can't skip them. Access goes to the app's service account with an object-level role on just that bucket, not a project-wide storage admin role. Data is encrypted at rest by default; if the customer or a regulation needs us to control the key, I'd use a customer-managed key in Cloud KMS. I'd turn on Data Access audit logs for Cloud Storage in that project so reads are recorded, and use versioning or a retention policy so a bad delete can be recovered. When a user needs to download a document, the app hands out a signed URL that expires in minutes rather than making anything public."
Relying on nobody knowing the bucket name, or making objects public to share them with one customer.
Pattern: the backend checks the user, then returns a short-lived V4 signed URL for one object.
Upload: the browser sends the file straight to Cloud Storage with that URL.
Details: fix the method and content type, set CORS on the bucket, and name objects on the server.
"I'd use signed URLs. The browser asks my backend for an upload slot; the backend checks the user is allowed, picks the object name itself, and returns a V4 signed URL that allows a PUT to that one object for the next few minutes. The browser then uploads straight to Cloud Storage, so my servers never carry the bytes. The URL is signed for a specific method and content type, so it can't be reused to read or overwrite something else. The bucket needs a CORS rule allowing PUT from our web origin. One practical detail: on Cloud Run there's no private key file, so I pass the service account email and a fresh access token, and the signing happens through the IAM credentials API. That means the service account needs the Token Creator role on itself. For very large files or flaky connections, I'd use a resumable upload session instead of a single PUT."
from datetime import timedelta
import google.auth
from google.auth.transport.requests import Request
from google.cloud import storage
credentials, project = google.auth.default()
client = storage.Client(credentials=credentials, project=project)
def upload_url(bucket_name, object_name):
credentials.refresh(Request()) # fresh token for keyless signing
blob = client.bucket(bucket_name).blob(object_name)
return blob.generate_signed_url(
version='v4',
expiration=timedelta(minutes=15),
method='PUT',
content_type='application/pdf',
service_account_email=credentials.service_account_email,
access_token=credentials.token,
)
Making the bucket publicly writable, or streaming every upload through the application servers.
What: serverless data warehouse; you write SQL, Google runs the compute.
How: columnar storage separate from compute; queries run in parallel across many workers.
Wrong for: high-rate single-row reads and updates, like an app's transactional database.
"BigQuery is Google Cloud's serverless data warehouse. I load or stream data into tables, write standard SQL, and Google runs the query across a lot of machines in parallel, with no cluster for me to size. Storage and compute are separate, and data is stored by column, so a query that touches three columns of a wide table only reads those three. That makes it very good at scanning and aggregating huge tables for reporting, dashboards and analysis. Where it's the wrong tool is transactional work: an app that reads and updates single rows thousands of times a second, needs low-millisecond responses, or relies on many small transactions. For that I'd use Cloud SQL, AlloyDB, Spanner or Firestore, depending on scale and data model, and feed the data into BigQuery for analytics."
Proposing BigQuery as the backend database for a user-facing app with frequent single-row updates.
Pricing: on-demand charges by bytes processed; capacity pricing pays for slots instead.
Habits: select only needed columns, filter on the partition column; LIMIT doesn't cut the scan.
Guardrails: dry runs, maximum bytes billed, custom quotas, budgets and alerts.
"Under on-demand pricing, a query is charged by the bytes it processes, not by rows returned or time taken. Since storage is columnar, the columns I select decide most of that, so SELECT star on a wide table is expensive even with a LIMIT, because LIMIT doesn't usually reduce what's scanned. The big levers are selecting only needed columns and filtering on the partition column so whole partitions get skipped. Clustering helps further when I filter on clustered columns. Before running something big, a dry run tells me how many bytes it will read. For guardrails, I set maximum bytes billed on jobs so a runaway query fails instead of running, and custom daily quotas per user or per project. Teams with steady heavy use can move to capacity pricing, paying for reserved slots, which makes cost predictable but means queries share that capacity."
Believing LIMIT 10 makes a scan of a huge table cheap, or not knowing what drives the charge.
Partitioning: splits the table into segments by date, timestamp, ingestion time or integer range; filters skip whole segments.
Clustering: sorts data inside storage by up to four columns so filters and joins on them read less.
Together: partition by date, cluster by what people filter on next, require a partition filter.
"Partitioning splits a table into separate segments, most often by a date column. A query that filters on that column only reads the matching partitions, and I can see the saving in a dry run before it runs. Clustering sorts the data inside the table, or inside each partition, by up to four columns, so filters on those columns read fewer blocks. It suits high-cardinality columns like customer ID, where one partition per value would make no sense. For an orders table I'd partition by the order date and cluster by customer ID and maybe status, because most questions are about a date range for some customers. I'd also require a partition filter so nobody scans years of history by accident. And I'd avoid tiny partitions, like hourly on a small table, because too many small partitions hurt more than they help."
CREATE TABLE shop.orders
PARTITION BY DATE(created_at)
CLUSTER BY customer_id, status
OPTIONS (require_partition_filter = TRUE)
AS
SELECT * FROM shop.orders_raw;
Trying to give every customer ID its own partition, or writing queries that never filter on the partition column.
Source: INFORMATION_SCHEMA jobs views, qualified by the region the data lives in.
Measure: total bytes billed per query, plus who ran it.
Next: group by user or query text to find the repeat offenders.
"BigQuery keeps a history of jobs in INFORMATION_SCHEMA, and the jobs-by-project view lists every job in the project. The view is regional, so I qualify it with the region where the datasets live. I filter to finished query jobs from the last seven days, skip the parent row of multi-statement scripts so nothing is counted twice, and sort by total bytes billed, which is what drives on-demand cost. I include the user and the query text so I can see who ran it and what it was. In practice the top rows are often a scheduled query or a dashboard that refreshes too often with SELECT star over an unpartitioned table. Once I've found them, I'd group by user or by query text to see repeat offenders, since one query run every few minutes usually costs more than one huge ad hoc query."
SELECT
user_email,
job_id,
ROUND(total_bytes_billed / POW(1024, 4), 3) AS tib_billed,
creation_time,
query
FROM `region-us`.INFORMATION_SCHEMA.JOBS_BY_PROJECT
WHERE creation_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
AND job_type = 'QUERY'
AND state = 'DONE'
AND statement_type != 'SCRIPT' -- a script's child queries are listed too
ORDER BY total_bytes_billed DESC
LIMIT 20;
Having no idea where query history lives and planning to guess the cost from the billing total.
Locate: billing report by project and SKU to confirm it's query analysis, not storage.
Pinpoint: job history in INFORMATION_SCHEMA sorted by bytes billed, grouped by user and query.
Stop: pause the scheduled query or dashboard, fix filters or table design.
Guard: maximum bytes billed, custom quotas, budget alerts to a channel people watch.
"First I'd open the billing report, group by project and SKU, and confirm it's query analysis rather than storage or streaming inserts, and which project it's in. Then I'd go to the jobs view in INFORMATION_SCHEMA for that project and sort yesterday's queries by bytes billed, grouped by user and query text. Very often it's something new that repeats: a scheduled query, a dashboard refreshing every few minutes, or a pipeline that dropped its partition filter and now scans the full table each run. I'd pause that job or dashboard straight away and tell its owner. The fix is usually adding the partition filter, selecting fewer columns, or pointing the dashboard at a small summary table. Then I'd add guardrails: maximum bytes billed on scheduled jobs, custom daily quotas, and budget alerts. Budgets only notify, so I'd make sure someone actually receives them."
Waiting for the monthly invoice, or deleting data or datasets before finding which job caused the cost.
Delivery: at-least-once by default; unacked messages come back after the ack deadline.
Duplicates: make processing idempotent with a message or business key; exactly-once delivery exists for pull subscriptions.
Order: ordering keys with ordering enabled on the subscription, only where it's really needed.
Failures: dead-letter topic after a set number of attempts.
"Pub/Sub delivers each message at least once. A subscriber has to acknowledge a message within the ack deadline; if it doesn't, because it crashed or was slow, the message is sent again. So duplicates are normal, and I make processing idempotent: I use a business key like an order ID and do an upsert, or record processed IDs so a repeat does nothing. Pull subscriptions can also turn on exactly-once delivery, which cuts redelivery after a successful ack, but I'd still keep the handler safe. For order, messages aren't ordered by default. If I need order per customer, I publish with the customer ID as the ordering key and enable ordering on the subscription, which orders messages per key only. For messages that keep failing, I set a dead-letter topic after a few attempts so one bad message doesn't block or loop forever."
Assuming each message arrives exactly once and in publish order without any configuration or idempotency.
Dataflow: fully managed runner for Apache Beam; one model for batch and streaming, with autoscaling.
Dataproc: managed Spark and Hadoop clusters, plus a serverless option for Spark.
Choose: new streaming pipelines lean Dataflow; existing Spark jobs and skills lean Dataproc.
"Dataflow is a fully managed service that runs Apache Beam pipelines. I write the pipeline once and it can run as batch or streaming, and Dataflow handles the workers, autoscaling and much of the tuning. It's strong for streaming, with windows, watermarks and late data built into the Beam model, so a Pub/Sub to BigQuery pipeline is a natural fit. Dataproc is managed Spark and Hadoop. It gives me a cluster in a short time, or a serverless option for Spark batch jobs, so it's the easy path when a team already has Spark code or Spark skills, or is moving Hadoop work off its own servers. A common pattern is a short-lived cluster per job that's deleted when the job finishes, with data kept in Cloud Storage. So my rule is: new pipelines, especially streaming, lean Dataflow; existing Spark work leans Dataproc."
Describing Dataflow and Dataproc as the same thing, or rewriting working Spark jobs in Beam for no reason.
Ingest: a small collector on Cloud Run publishes events to a Pub/Sub topic.
Process: a BigQuery subscription for simple cases, or Dataflow for cleaning, dedupe and windows.
Store: a BigQuery table partitioned by event date and clustered on common filters.
Safety: dead-letter bad events, monitor backlog and pipeline lag.
"The browser sends events to a small collector service on Cloud Run, which validates them and publishes to a Pub/Sub topic. Pub/Sub absorbs spikes, so a traffic burst doesn't take anything down. If events only need to land as they are, a BigQuery subscription can write them straight into a table with no pipeline code. If I need cleaning, enrichment, deduplication or rolling counts, I'd put a streaming Dataflow pipeline in between, reading from Pub/Sub, dropping duplicates by event ID, and writing to BigQuery. Rows that fail parsing go to a dead-letter table so I can fix and replay them. The events table is partitioned by event date and clustered by things like page or user, so analysts' queries stay cheap. I'd alert on subscription backlog age and pipeline lag, and handle late events by allowing some lateness in the windows."
Having the website write directly to BigQuery with no buffer, or ignoring duplicates and malformed events.
Signals: built-in metrics, plus the Ops Agent on VMs for in-guest memory, disk and app logs.
Alerts: uptime checks and alerting policies on errors and latency, sent to a real channel.
Logs: structured logs, log-based metrics, and sinks to route or keep logs longer.
"Most services send metrics to Cloud Monitoring on their own, like request count and latency for Cloud Run or CPU for VMs. On Compute Engine I'd install the Ops Agent, because proper in-guest memory and disk usage and application logs come from it. Then I'd set up an uptime check that hits a real health endpoint from outside, and alerting policies on what users feel: error rate and latency, not only CPU. Alerts go to a channel someone actually watches. For logs, I'd have the app write structured JSON so fields like request ID and severity are searchable in Cloud Logging. If a specific log line matters, say a failed payment, I make a log-based metric and alert on it. Finally, I'd use log sinks to send logs to BigQuery or Cloud Storage for longer retention, and exclusion filters to drop noisy logs I don't need."
Checking the console by hand when users complain instead of having alerts on errors and latency.
Where: Cloud Audit Logs, Admin Activity type, in Logs Explorer for that project.
Filter: the Compute Engine service, the delete method and the instance name.
Know: Admin Activity logs are always on; Data Access logs are mostly off unless enabled.
"Deleting a VM is an admin action, so it shows up in the Admin Activity audit logs, which are always on and can't be turned off. I'd open Logs Explorer in that project, pick the activity audit log, and filter for the Compute Engine service and the instance delete method, plus the instance name. The entry shows the principal who called it, the time and the source IP, and whether it came from the console, gcloud or a service account in some automation. If it was a service account, I'd follow the trail to find which pipeline or person used it. One thing worth knowing: Data Access logs, which record reads of data, are off by default for most services, so for questions like who read a bucket I'd need them turned on ahead of time. Afterwards I'd look at why one person could delete production at all."
Not knowing audit logs exist, or assuming every read and write is logged by default.
Front door: global external Application Load Balancer with one anycast IP, plus Cloud CDN and Cloud Armor.
Compute: stateless app in two regions, as Cloud Run services or regional instance groups.
Data: regional high availability plus a cross-region replica, or a multi-region database.
Practice: state RPO and RTO, and rehearse the failover.
"I'd put a global external Application Load Balancer in front, which gives one anycast IP and sends users to the nearest healthy backend. Behind it I'd run the app in two regions, either as Cloud Run services through serverless network endpoint groups or as regional managed instance groups, so a zone failure is already covered inside each region. If one region fails its health checks, traffic shifts to the other. The app is stateless: sessions in a shared store, files in a dual-region or multi-region Cloud Storage bucket. The database is the hard part. With Cloud SQL, I'd use a high-availability instance in the main region and a cross-region read replica I can promote, accepting a small data loss window and a manual step. If the business needs writes in both regions with no loss, that points to Spanner. Then I'd agree the recovery targets and actually rehearse the failover."
Saying the app is multi-region while the only database sits in one region with no tested failover.
Situation: what users saw and how you found out.
Trace: the logs, metrics or audit entries that narrowed it down.
Fix: the immediate fix, the root cause, and what you changed so it doesn't recur.
"At my last company, our order API on Cloud Run started returning errors for a few minutes every morning. The service metrics showed a spike in container instance count and latency right when a batch export kicked off. In the logs, the failing requests were all timeouts talking to Cloud SQL, and the database metrics showed connections hitting the limit. The export job and the burst of new API instances were each opening their own connection pools, and together they exhausted the database. As a quick fix I capped the maximum instances on the API and shrank each instance's pool, which stopped the errors that day. Then I moved the export to read from a replica and added an alert on database connections nearing the limit. The morning errors went away, and I wrote up the limits so the next service sized its pool on purpose."
A story where the fix was restarting things until it worked, with no root cause or follow-up.
Why: the problem with the old setup.
Changes: what the app needed, like config, state, identity or networking.
Cutover: how you shifted traffic safely and what you measured after.
"In my last role we had a set of internal APIs on a few Compute Engine VMs that someone patched by hand, and deployments meant SSH and a script. I moved them to Cloud Run. The code barely changed, but the app did: it had to listen on the port Cloud Run provides, read config from environment variables and Secret Manager instead of files on disk, and stop writing uploads to local disk, which I moved to Cloud Storage. Each service got its own service account with narrow roles instead of sharing the VM's account. For cutover I put both old and new behind the same load balancer and shifted traffic in steps, watching error rate and latency, with a one-line rollback ready. Deploys went from a manual job to a pipeline run, and we stopped patching servers altogether."
A big-bang cutover with no rollback plan and no measure of whether things got better.
Found: what the access looked like and why it was risky.
Method: usage data, role recommendations, groups, a staged rollout.
Result: what changed and how you kept it from drifting back.
"When I joined my last team, about twenty people had Editor on the production project, plus a couple of old service accounts with Owner. Nobody could say who needed what. I didn't just remove it, because that would break someone's work on day one. I used the IAM role recommendations, which look at permissions actually used, and the audit logs to see who did what over recent months. Then I created a few groups, like deployers and read-only support, with predefined roles that matched, and moved people into them after telling each lead what was changing. The Owner service accounts turned out to be for a retired pipeline, so I disabled them first, waited, then deleted them. Access dropped to a handful of named roles, and I added a check in our infrastructure code review so direct user grants on production get flagged."
Revoking everyone's access at once with no data or warning, or leaving it alone because it might break something.
ClapAssist is an AI interview assistant for Mac and Windows. It listens to the interview on your computer and shows you what to say, in short lines you can read while you talk. Your resume and notes are never stored on our servers. It stays out of screen share on every plan; only you can see it.