From time-series tasks
to learned representations
and foundation models.
Lecture 2 · 90 minutes
Intro to ML · Time Series II
Core ideaWhat is the signal telling us?
Match the output contract and evaluation protocol to the operational decision.
Core idea
Serega has a clever forecaster. Is the pump healthy?
It predicts the next reading accurately.
Does that prove normal operation?
Keep this question open until the final experiment.
Core idea
Five tasks. Five different outputs
Choose the required output before choosing the model.
Worked example
Classification: a label for an observed window
One observed epoch → one class. Bandpower + logistic regression; MiniROCKET features or learned EEGNet features.
Worked example
Forecasting: predict values that have not arrived
History → future horizon. Compare every model at the same issuance time and forecast horizon.
Core idea
Anomaly detection: unusual relative to what?
A score becomes a decision only after thresholding. Define whether the label covers an onset or the entire abnormal state.
Worked example
Segmentation: label the timeline, or find its boundaries
State labeling: HMM or temporal CNN. Change-point detection: PELT / ruptures.
Worked example
Imputation: what information may cross the gap?
Linear interpolation → Kalman methods → SAITS. Offline filling may use right context; online filling may not.
Core idea
The architecture expresses an assumption
Reference detail
A causal TCN grows its field of view
Dilation skips input positions while retaining a local kernel. Future values must stay outside the receptive field.
Core idea
Memory and attention answer different questions
Both can learn long dependencies; they store and access the available history differently.
Bridge · From pump sensors to EEG
Core ideaEEG: time is only one of the axes
Read a neural time-series model by following its axes and operations.
Same axis-reading skill, different target: one movement label for an observed EEG epoch.
Worked example
A classical EEG pipeline already has useful structure
EEGNet makes a related sequence of temporal and spatial operations learnable end to end.
Worked example
A temporal kernel has a frequency response
These kernels are chosen by hand. EEGNet learns temporal weights; we can inspect them with the same Fourier tools.
Pump investigation 6/8 · Can a filter extract a useful feature?
Worked exampleOne convolution, computed by hand
Core idea
Temporal first. Spatial second.
Each temporal feature gets its own spatial projections. Depthwise means we do not mix different temporal filters here.
Worked example
Separable convolution: refine, then mix
Focus: Distinguish per-map temporal refinement from pointwise feature-map mixing.
Worked example
Follow one EEG epoch through the network
Focus: Follow which EEGNet operations change maps, electrodes and time.
Core idea
An EEG score is only as honest as its split
Bridge · Back to the pump
Core ideaWhat changes when we pretrain once?
Explain pretrained forecasting and evaluate a complete detector under a fixed alarm budget.
Local models fit each series. Global models share weights. A pretrained forecaster reuses weights on a new task.
Worked example
Chronos-2: a modern forecasting model we can read
autogluon/chronos-2-small · 6 blocks · 8 heads · hidden size 512 · 13 quantiles
Core idea
A forecasting task is data plus a visibility contract
Worked example
Scaling keeps units from dominating the network
z = asinh((x − μ) / σ) → x̂ = μ + σ sinh(ẑ)
Core idea
A missing value and a measured zero are different
The schedule is hypothetical. These are input-construction numbers, not a measured forecast.
Worked example
A patch becomes a vector, without value quantization
Focus: Explain how values, time and an observation mask become one patch representation.
Core idea
Time attention reads within one variable
Chronos-2 uses an encoder. Its time attention is bidirectional across the supplied context and future placeholders.
Core idea
Group attention reads across related variables
At one patch index, series with the same group ID exchange information. Different group IDs block that exchange.
Worked example
Quantile loss makes over- and underprediction cost different
At q = 0.9: underpredict by 1 → loss 0.9; overpredict by 1 → loss 0.1.
Core idea
Pretraining teaches transferable forecasting structure
Pump investigation 7/8 · Has the fault ended?
Worked exampleA forecast can become comfortable with a fault
Core idea
A detector is a chain of choices
Compare complete detection systems, then inspect which stage explains their differences.
Define the denominator
Worked exampleMacro and pooled answer different questions
In this lab, AP and F1 give every run one vote; normal alarm time gives every normal timestamp one vote.
Worked example
Rolling features help this model on validation
Same Isolation Forest: 200 trees. Raw sensors → 48 causal sensor, difference and rolling features. Validation only.
Worked example
Averaging helps here; max inherits the saturated tail
Fixed validation run: valve1/8. Shading = abnormal state. Mean fusion averages three calibrated scores.
Worked example
Better forecasts do not guarantee better state detection
Chronos reduces point-forecast error on these validation runs; its selected residual detector still misses most faulty timestamps.
Worked example
Read the horizon and check the interval
The nominal 80% Chronos interval covers about 76% here. Read coverage together with width.
Worked example
Read the score, then read the actual alarm
A high score is not an alarm. On this fixed run, the mean ensemble issues no alerts at its frozen cutoff.
Core idea
Serega can read the forecast. You define the question.
Specify the target.
Follow the axes and the masks.
Compare with an honest baseline.
In the lab: explain one success and one failure before changing the model.
