Compute Engine, Cloud Run, BigQuery, Cloud Storage, Pub/Sub, IAM, and GKE — Google Cloud Platform.
# ── Install & Setup ──
curl https://sdk.cloud.google.com | bash
gcloud init
gcloud config set project my-project-id
gcloud config set compute/region us-central1
# ── Compute Engine VM ──
gcloud compute instances create my-vm \
--machine-type e2-medium \
--image-family ubuntu-2204-lts \
--image-project ubuntu-os-cloud \
--boot-disk-size 50GB \
--tags http-server \
--metadata startup-script='#!/bin/bash
apt-get update && apt-get install -y nginx
systemctl start nginx'
# Manage VM
gcloud compute instances list
gcloud compute ssh my-vm
gcloud compute instances stop my-vm
gcloud compute instances start my-vm
gcloud compute instances delete my-vm
# ── Cloud Run (Serverless Containers) ──
gcloud run deploy my-service \
--source . \
--region us-central1 \
--allow-unauthenticated \
--memory 512Mi \
--cpu 1 \
--min-instances 0 \
--max-instances 100
# List services
gcloud run services list
gcloud run services describe my-service| Family | Type | vCPUs | RAM | Use Case |
|---|---|---|---|---|
| e2 | e2-micro | 2 (shared) | 1 GB | Dev, small workloads |
| e2 | e2-medium | 2 (shared) | 4 GB | Web servers |
| n2 | n2-standard-4 | 4 | 16 GB | General purpose |
| n2d | n2d-standard-8 | 8 | 32 GB | AMD EPYC,性价比 |
| c2 | c2-standard-4 | 4 | 16 GB | Compute-intensive |
| m2 | m2-ultramem-208 | 208 | 5872 GB | Memory-intensive |
| Feature | Details |
|---|---|
| Scaling | 0 to 1000+ instances automatically |
| Concurrency | Up to 1000 requests/instance |
| Max Timeout | 60 min (configurable) |
| CPU Allocation | Always allocated (1-8 vCPUs) |
| Memory | 128 MiB to 32 GiB |
| Pricing | Pay per request + vCPU-seconds |
| Cold Start | Typically <200ms |
-- ── Create Dataset ──
CREATE SCHEMA IF NOT EXISTS my_project.analytics
OPTIONS (description='Analytics dataset', location='US');
-- ── Create Partitioned Table ──
CREATE OR REPLACE TABLE my_project.analytics.events (
event_id STRING NOT NULL,
user_id STRING,
event_type STRING,
event_data JSON,
created_at TIMESTAMP NOT NULL
)
PARTITION BY DATE(created_at)
CLUSTER BY user_id, event_type
OPTIONS (
description='User events table',
partition_expiration_days=365
);
-- ── Insert Data ──
INSERT INTO my_project.analytics.events (event_id, user_id, event_type, created_at)
VALUES
('e1', 'u100', 'page_view', TIMESTAMP('2024-01-15 10:00:00')),
('e2', 'u100', 'click', TIMESTAMP('2024-01-15 10:01:00')),
('e3', 'u200', 'purchase', TIMESTAMP('2024-01-15 11:00:00'));
-- ── Aggregate Queries ──
SELECT
DATE(created_at) AS event_date,
event_type,
COUNT(*) AS event_count,
COUNT(DISTINCT user_id) AS unique_users
FROM my_project.analytics.events
WHERE created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
GROUP BY 1, 2
ORDER BY 1 DESC, 3 DESC;
-- ── Window Functions ──
SELECT
user_id,
event_date,
event_count,
SUM(event_count) OVER (
PARTITION BY user_id ORDER BY event_date
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
) AS running_total
FROM (
SELECT
user_id,
DATE(created_at) AS event_date,
COUNT(*) AS event_count
FROM my_project.analytics.events
GROUP BY 1, 2
);| Feature | Description |
|---|---|
| PARTITION BY | Partition table by date column |
| CLUSTER BY | Cluster by frequently filtered columns |
| UNNEST() | Flatten ARRAY and STRUCT data |
| ANY_VALUE() | Aggregate without GROUP BY all columns |
| QUALIFY | Filter on window function results |
| FOR SYSTEM_TIME AS OF | Time travel queries |
| SAFE_DIVIDE() | Division that returns NULL on /0 |
| FORMAT_DATE() | Format timestamps to strings |
| Model | Cost | Details |
|---|---|---|
| On-demand | $6.25/TB processed | Pay per query, 1 TB free/mo |
| Flat-rate | Fixed monthly | $10k (100 slots), committed |
| Storage | $0.02/GB/month | Active storage |
| Long-term | $0.01/GB/month | After 90 days, auto-tiered |
| Streaming | Free inserts | Up to 500 rows/sec free |
# ── Cloud Storage ──
gsutil mb gs://my-bucket-unique-123/
gsutil cp ./file.txt gs://my-bucket-unique-123/
gsutil cp -r ./dist/* gs://my-bucket-unique-123/
gsutil ls gs://my-bucket-unique-123/
gsutil rm gs://my-bucket-unique-123/old-file.txt
# Set lifecycle policy
gsutil lifecycle set lifecycle.json gs://my-bucket-unique-123/
# Make public
gsutil iam ch allUsers:objectViewer gs://my-bucket-unique-123/
# ── Pub/Sub ──
gcloud pubsub topics create my-topic
gcloud pubsub subscriptions create my-subscription --topic my-topic
# Publish message
gcloud pubsub topics publish my-topic --message "Hello World"
# Pull messages (Node.js)
# const {PubSub} = require('@google-cloud/pubsub');
# const pubsub = new PubSub();
# const subscription = pubsub.subscription('my-subscription');
# subscription.on('message', message => {
# console.log('Received:', message.data.toString());
# message.ack();
# });| Class | Latency | Cost | Use Case |
|---|---|---|---|
| Standard | Low | Standard | Frequently accessed data |
| Nearline | Low | Lower | 30-day min, infrequent access |
| Coldline | Low | Even lower | 90-day min, rare access |
| Archive | Milliseconds | Lowest | 365-day min, archival |
| Concept | Description |
|---|---|
| Topic | Named resource for messages |
| Subscription | Named connection to a topic |
| Publisher | Sends messages to a topic |
| Subscriber | Receives messages from a subscription |
| Acknowledgment | Confirms message processing |
| Dead Letter | Failed messages go to DLQ |
# ── Create GKE Cluster ──
gcloud container clusters create my-cluster \
--region us-central1 \
--num-nodes 3 \
--machine-type e2-medium \
--enable-autoscaling --min-nodes 1 --max-nodes 10 \
--enable-autorepair \
--enable-autoupgrade \
--enable-ip-alias
# Get credentials
gcloud container clusters get-credentials my-cluster --region us-central1
# Deploy application
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml
kubectl get pods -w
kubectl get services
# ── IAM ──
# Create service account
gcloud iam service-accounts create my-app-sa \
--display-name "My App Service Account"
# Grant roles
gcloud projects add-iam-policy-binding my-project-id \
--member="serviceAccount:my-app-sa@my-project-id.iam.gserviceaccount.com" \
--role="roles/storage.objectViewer"
# Create key (for GKE workload identity preferred)
gcloud iam service-accounts keys create key.json \
--iam-account=my-app-sa@my-project-id.iam.gserviceaccount.comCloud Run is fully managed serverless container platform. No cluster management, scales to zero, pay per request. Best for stateless microservices, APIs, event-driven workloads.GKE provides full Kubernetes control. You manage nodes, networking, and configuration. Best for complex workloads, custom scheduling, stateful apps, or when you need full Kubernetes API access.
Partition by date for time-series data. Cluster by frequently filtered columns (not high-cardinality). Use nested/repeated fields instead of joins where possible. Avoid SELECT * — query only needed columns. Use materialized views for frequently run queries. Use streaming inserts for real-time data. Monitor with INFORMATION_SCHEMA views for slot utilization and query history.