Machine Learning on AWS

Data foundations and storage

Model quality is capped by data quality. Data may be structured (tables), semi-structured (JSON, logs) or unstructured (images, audio, text), and it is judged on volume, velocity and variety, plus how trustworthy it is. A data warehouse holds curated, schema-first data for reporting, a data lake stores raw data of any shape cheaply for later use, and a lakehouse blends both. ETL (or ELT) pipelines move and reshape data between them.

On AWS, S3 is the usual lake foundation, with columnar formats such as Parquet and ORC preferred for analytics because they scan less data than CSV or JSON. Partition by fields you filter on (such as date), apply lifecycle rules to age data into colder classes, and encrypt buckets. Glue crawlers populate a Data Catalog so Athena can query S3 with SQL and other services can discover schemas. For training jobs that need fast shared access, EFS or FSx for Lustre can complement S3; block volumes suit single-instance work.

Ingestion: Kinesis, Firehose and MSK

• Kinesis Data Streams ingests ordered records into shards, with on-demand or provisioned capacity, and lets multiple consumers read the same data within a retention window.
• Amazon Data Firehose is a managed delivery pipeline: it buffers records, can transform them with a function or convert formats, and writes them to S3, a warehouse, a search domain or an HTTP endpoint. It is near-real-time rather than real-time and needs no consumer code.
• Managed Service for Apache Flink runs stream-processing applications such as windowed aggregations, enrichment and anomaly detection.
• MSK is managed Apache Kafka. Choose it when you already use Kafka clients and ecosystem tooling, or need Kafka-specific features such as log compaction; choose Kinesis for tighter AWS integration and simpler operations.

A common pattern is producers writing to a stream, Firehose landing raw events in S3, and batch or streaming jobs deriving features from them.

Processing and feature engineering

EMR runs big-data frameworks, chiefly Apache Spark, on managed clusters (on EC2, on Kubernetes, or serverless). Use short-lived clusters for scheduled jobs and keep data in S3 so compute is disposable. Glue offers serverless ETL, a visual studio, data-quality rules and DataBrew for no-code cleansing, including masking personal data.

Feature engineering turns raw columns into signals a model can use:

• Missing values: drop rows, impute with mean, median or a model, or add an "is missing" flag.
• Imbalanced classes: resample, generate synthetic examples or weight the loss.
• Outliers: inspect first, then cap, transform or remove with justification.
• Encoding: one-hot or ordinal for categories; scaling or normalisation for numeric ranges; binning and log transforms for skewed values; TF-IDF for text.
• Shuffling before splitting avoids ordering bias, except for time series, which should be split chronologically.

Always split into training, validation and test sets before fitting any transformation, and fit scalers only on training data to prevent leakage.

The SageMaker workflow

1. Explore in Studio notebooks; Data Wrangler and Canvas provide low-code preparation.
2. Label with Ground Truth, which combines human annotators and automation.
3. Store features in the Feature Store so training and inference share identical definitions.
4. Train with a built-in algorithm (linear learner, XGBoost, k-means, image classification and others), a prebuilt framework container or your own container. Use spot capacity with checkpointing to cut cost, and distributed training libraries for large models.
5. Tune with automatic model tuning, which searches hyperparameters using random, Bayesian or similar strategies; keep the search space small and meaningful.
6. Track runs with Experiments or managed MLflow, inspect training with Debugger, and register approved models in the Model Registry.
7. Deploy and monitor, orchestrating the steps with Pipelines for repeatability.

Autopilot, now surfaced through Canvas, automates algorithm selection for tabular data and is a quick baseline.

Evaluation metrics

For classification, start with the confusion matrix of true and false positives and negatives.

Metric	Meaning
Precision	Of predicted positives, how many were right. Prioritise when false alarms are costly.
Recall	Of real positives, how many were found. Prioritise when misses are costly, such as fraud or disease.
Specificity	Of real negatives, how many were correctly rejected.
F1	Harmonic mean of precision and recall; useful for imbalanced data.
AUC-ROC	Ranking quality across all thresholds; the precision-recall curve is more informative when positives are rare.
RMSE / MAE / R-squared	Regression error and explained variance; RMSE punishes large errors more than MAE.

Plain accuracy misleads on skewed data: a model that always predicts the majority class can score highly while being useless.

Overfitting and underfitting

Underfitting (high bias) means the model is too simple and performs poorly even on training data. Overfitting (high variance) means it memorised noise: excellent training scores, much worse validation scores. Compare the two curves to diagnose which you have.

• Fix underfitting with more features, a more expressive model, longer training or less regularisation.
• Fix overfitting with more or better data, simpler models, L1 or L2 regularisation, dropout, early stopping, cross-validation and ensembles such as bagging.
• For neural networks, tune learning rate and batch size, pick suitable activation functions (ReLU variants help avoid vanishing gradients), and normalise inputs.

Deployment options

Mode	Use when
Real-time endpoint	Low-latency, steady request flow; supports autoscaling and multiple variants for A/B tests.
Serverless inference	Spiky or intermittent traffic where idle capacity is wasteful, accepting cold starts.
Asynchronous inference	Large payloads or long processing, with queued requests.
Batch transform	Scoring a whole dataset offline, no persistent endpoint.
Edge	Running compiled models on devices when connectivity or latency demands it.

Release safely with blue/green or canary traffic shifting and automatic rollback on alarms.

Monitoring and safeguards

Models degrade as the world changes. Model Monitor captures endpoint traffic and compares it to a baseline to detect data-quality problems, data drift, model-quality decay and shifts in bias or feature attribution. Clarify reports bias before and after training and explains predictions using SHAP values and partial dependence plots. Wire alerts to retraining pipelines.

Secure the stack with least-privilege roles, encryption with managed keys, VPC-only training and endpoints, network isolation for sensitive jobs, and audit logs. Keep humans in the loop for low-confidence predictions and document datasets, intended use and known limitations.