Descriptive and predictive modeling — the Company B ruck march problem
This is unofficial review material prepared by the course CIC for the study session. It is not the WPR and is not drawn from any secured assessment. Coverage follows the published lesson objectives for Lessons 1–21 and the conventions of Homeworks 1 and 2. Per the course director, the emphasis is conceptual understanding and precise mathematical interpretation, not Excel mechanics.
Course conventions used here: the outlier screen is |z| > 2.5 (HW1 Q16); residual plots are drawn against fitted values (HW2 Q11); KNN is a classification method decided by majority vote (HW2 §5); and the starting k is √ntrain rounded to the nearest odd integer (HW2 Q22).
Situation. You are the operations officer for Company B, 2nd Regiment. The company runs a 12-mile ruck march at the end of every term, and the commander wants to use last term's data to (1) describe how the company actually performed and (2) predict how individual cadets will do this term, so at-risk cadets can be put on a remedial training plan. All ruck times are in minutes; all cadets carried a 35-pound rucksack over the same route. The Army standard for the 12-mile foot march is 180 minutes — finish at or under it and you earn a GO.
Lessons 1–3, 8. Framing a question, population versus sample, classifying variables, partitioning data, and telling an unusual observation from a bad one.
Label each question below. Then, for (b) only, state in one sentence what makes it that type.
(a) Descriptive (b) Predictive (c) Descriptive
(b) is predictive because it asks for an estimate of an unobserved outcome for a cadet who is not in the dataset. Descriptive questions summarize data you already have; predictive questions produce a value for a new case.
(a) The commander's question is about this term's cadets, but your data is last term's 64 records. Identify the population of interest and state whether your data is the full population or a sample. How do you know?
(b) Complete the data dictionary. Give the variable type (categorical, or numerical and then discrete or continuous) and the units, if any.
| Column | Example | Type | Units |
|---|---|---|---|
| Platoon | 3rd | ||
| Ruck time | 184.5 | ||
| Prior ruck marches completed | 7 | ||
| Met the 12-mile standard | GO |
(c) In one sentence, why should the commander require a data dictionary before anyone analyzes this roster?
(a) The population is all Company B cadets who will run this term's march — the group the commander must decide about. The 64 records are a sample, and not even a random one: they are last term's cadets, a different group in a different term. Every inference then carries the assumption that last term's cadets resemble this term's.
(c) A data dictionary fixes the meaning, units, and allowed values of every column, so two analysts working the same file compute the same thing and nobody silently treats minutes as hours or averages a platoon number.
A cadet builds a model on all 64 company records, reports that it fits those 64 records very well, and recommends it to the commander. State what is wrong with that procedure and what the cadet should have done instead. Be specific about what the reported error is actually measuring.
The reported error is training error: it measures how well the model reproduces the very data used to build it, not how well it predicts a cadet it has never seen. A flexible enough model drives training error to zero and can still be useless out of sample. The cadet should have partitioned into a training and a test set (roughly 80/20), fit on the training set only, and reported performance on the held-out test set — the only honest estimate of performance on this term's cadets.
For the commander's remedial-training problem, state the decision, the objective, and one assumption the model requires.
Lessons 2–7. Center, spread, quartiles, standardizing, choosing the right summary, correlation, and captioning a figure.
Questions 5–9 use the eight recorded 12-mile ruck times, in minutes, from 1st Platoon, in ascending order:
| x1 | x2 | x3 | x4 | x5 | x6 | x7 | x8 |
|---|---|---|---|---|---|---|---|
| 168 | 172 | 175 | 175 | 180 | 184 | 190 | 236 |
Calculate the mean and the median. Show your work.
Sum = 1480. Mean = 1480 ÷ 8 = 185 minutes
n = 8 is even, so the median averages the 4th and 5th ordered values: (175 + 180) ÷ 2 = 177.5 minutes
The mean sits 7.5 minutes above the median — the outlier at 236 doing work.
Calculate the sample standard deviation s. Show the squared deviations.
s = (two decimal places)
| xi | 168 | 172 | 175 | 175 | 180 | 184 | 190 | 236 |
|---|---|---|---|---|---|---|---|---|
| deviation | −17 | −13 | −10 | −10 | −5 | −1 | 5 | 51 |
| squared | 289 | 169 | 100 | 100 | 25 | 1 | 25 | 2601 |
Σ = 3310 → s2 = 3310 ÷ 7 = 472.86 → s = 21.75 minutes
One observation contributes 2601 of 3310 — 79% of the total squared deviation from a single cadet. That is why s is not resistant — and why the boxplot in Question 9, which is built from quartiles rather than from s, tells a steadier story about this platoon.
Use your own x̄ from Question 5 and s from Question 6. Follow-on credit applies, so carry your values forward and continue.
(a) z8 = (236 − 185) ÷ 21.75 = 2.34 z1 = (168 − 185) ÷ 21.75 = −0.78
(b) Neither observation is flagged. The screen is |z| > 2.5; the slowest cadet reaches only 2.34, and the fastest is well inside at 0.78.
What the screen does and does not establish: it is a rule of thumb for selecting values to review, not a test of validity. Clearing it does not certify a value as correct, and exceeding it does not prove a value is wrong — a 34-minute ruck time would be caught by any screen, while a plausible-looking mistyped 185 would pass every screen there is. Context, not the threshold, decides.
Direction matters too: a positive z on ruck time means slower, not better.
The histogram shows all 64 recorded ruck times in Company B.
(a) Circle the measure of center you would report as the "typical" ruck time, and justify it in one to two sentences using the language of the course.
(b) Without computing anything, state whether the mean is larger or smaller than the median here, and why.
(a) Median. The distribution is clearly right-skewed — unimodal, with a long tail of slow times out to 240 minutes and no matching tail on the fast side. The mean is pulled toward that tail, so it overstates what a typical cadet runs; the median is resistant to those extreme values.
(b) Mean > median. The long upper tail drags the mean above the median — consistent with 1st Platoon, where 185 > 177.5.
Below is a boxplot of the eight 1st Platoon ruck times from Questions 5–9.
(a) Read the five-number summary off the figure: the box edges, the line inside the box, and the whisker ends. State the IQR.
(b) What does the boxplot show that the histogram does not, and what does the histogram show that the boxplot does not?
(a) Q1 ≈ 173.5, median 177.5, Q3 ≈ 187; whiskers at 168 and 190, with the 236-minute cadet plotted separately past the fence. The box width is the IQR, ≈ 13.5 minutes — read it straight off the axis.
(b) The boxplot gives the five-number summary and applies the 1.5 × IQR fence, so it identifies the 236-minute cadet as a flagged point explicitly and makes the quartiles readable at a glance. The histogram shows the shape — modality, where the bulk sits, how the tail decays — which the boxplot hides entirely. A boxplot cannot reveal a bimodal distribution; a histogram cannot give you exact quartiles.
The right answer to "which is better" is that they answer different questions. For the commander's read-ahead the histogram tells the story; for flagging individual cadets the boxplot does.
Excel returns the correlations below, computed on all 64 records. Every cadet carried a 35-pound ruck; the load column varies only because of small differences in how canteens were filled.
| Variable | r |
|---|---|
| Weekly training mileage (mi/wk) | −0.92 |
| 2-mile run time (min) | 0.78 |
| Body weight (lb) | 0.15 |
| Ruck load (lb) | 0.04 |
(a) Weekly training mileage, |r| = 0.92. The negative sign means cadets who train more miles per week tend to post lower ruck times — they finish faster. A linear model on it is reasonable (0.92 is a strong linear association and a scatterplot supports a straight-line form) and necessary (the commander needs a predicted time for cadets not yet measured, which a summary statistic cannot give).
(b) No. Observational data; nobody assigned cadets to a training volume. A plausible third variable is baseline fitness — fitter cadets both choose to train more and ruck faster, producing this correlation with no causal effect of the training itself.
(c) No. Either reason earns full credit:
You are preparing a one-page read-ahead for the battalion commander, who has no statistics background and must decide whether to fund a remedial ruck program. Write a caption for Figure 1. A caption has a title and then, in three to five sentences, (1) describes the figure and (2) explains what the reader should take away and why it matters for the decision.
Model response. Figure 1. Most of Company B rucks near 180 minutes, but a slow tail of roughly a dozen cadets runs past 200. The histogram shows the distribution of 12-mile ruck times for all 64 cadets in Company B, binned in 10-minute intervals. The distribution is right-skewed: it peaks between 170 and 190 minutes, where more than half the company falls, then trails off through a long upper tail reaching 240 minutes. Because of that tail the company average (185 minutes) is slower than the typical cadet's time (177.5 minutes), so quoting the average alone understates how most of the company performs and hides the group actually at risk. The decision is not about the company average — it is about the dozen or so cadets past 200 minutes, and a remedial program should be sized to them.
On Gate 2 the caption was worth 10 of 25 points and was one of the two most-missed items. The grader's note on a 9/10 answer was that the cadet described the model as only over-predicting in a range where the residuals actually fan both ways — if you write about a residual plot, name the equal-variance problem precisely rather than describing one direction of error.
Where cadets lose points: no title; describing the axes but never stating a takeaway; writing for a statistician instead of for the commander who has to decide something.
Lessons 9–16. Reading a fitted line, interpreting slope, intercept and R2, residual diagnostics against fitted values, and out-of-sample error.
Six cadets form the training set. x is average weekly training mileage, y is the 12-mile ruck time.
| Cadet | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| x — mileage (mi/wk) | 5 | 10 | 15 | 20 | 25 | 30 |
| y — ruck time (min) | 210 | 195 | 189 | 183 | 177 | 180 |
Do not use the data table — read Figure 3.
(a) At x = 10 the line reads 198; at x = 30 it reads 174. Accept 197–199 and 173–175.
(b) b1 = (174 − 198) ÷ (30 − 10) = −1.2; b0 = 198 + 1.2(10) = 210
ŷ = 210 − 1.2x
Two habits: read the line, never a nearby data point; and put rise over run in that order — inverting gives −0.83, the usual slip.
Excel reports the trendline as ŷ = 210 − 1.2x with R2 = 0.854. Use these for the rest of Part III.
(a) Each additional mile per week of training is associated with a 1.2-minute decrease in predicted 12-mile ruck time. Say associated with, not causes — observational data.
(b) The intercept says a cadet who trains 0 miles per week is predicted to ruck 12 miles in 210 minutes. Note that x = 0 sits outside the training range of 5–30 mi/wk, so the intercept is an anchor for the line rather than a claim about a real cadet — Question 14 develops that point.
The battalion S3 likes your model and proposes using it to identify cadets for a high-mileage training pilot. He asks what ruck time the model predicts for a cadet who trains 60 miles per week.
(a) ŷ = 210 − 1.2(60) = 210 − 72 = 138 minutes
(b) No — this is extrapolation and the prediction is not trustworthy. Three things to say, any two of which earn full credit:
What to do instead: collect data on cadets who actually train at that volume, or restrict the model's use to the 5–30 mi/wk range and say so explicitly when reporting it.
Complete the sentence, then state one thing R2 does not tell you.
Approximately % of the variation in is explained by its linear relationship with .
Strong, moderate, or weak?
Approximately 85.4% of the variation in 12-mile ruck time is explained by its linear relationship with weekly training mileage. Describe it as strong.
What it does not tell you: that the linear model is the right shape (Q17 shows it is not), that the model predicts well out of sample (Q19 shows it degrades), or that mileage causes the difference. R2 is computed on the training data only.
(a) Complete the four empty rows using ŷ = 210 − 1.2x.
| x | y actual | ŷ predicted | e residual |
|---|---|---|---|
| 5 | 210 | ||
| 10 | 195 | 198 | −3 |
| 15 | 189 | ||
| 20 | 183 | 186 | −3 |
| 25 | 177 | 180 | −3 |
| 30 | 180 |
(b) Compute the training MAE from all six residuals.
(c) Why does MAE use absolute values? What would happen if you averaged the six signed residuals?
(a) x = 5: ŷ = 204, e = +6 x = 15: 192, −3 x = 30: 174, +6
(b) Residuals +6, −3, −3, −3, −3, +6 → MAE = 24 ÷ 6 = 4.0 minutes. On the data it was fit to, the model misses by about 4 minutes on average.
(c) The signed residuals sum to zero — a property of the least-squares line, not a coincidence — so their average is 0 and would report a perfect model no matter how badly it fit. Absolute values stop positive and negative misses from cancelling.
(a) Plot the six residuals from Question 16 against the fitted values ŷ.
(b) Assess each assumption, citing evidence from your plot: linearity, constant variance, outliers.
Plotting note: because the slope is negative, the largest fitted value (204) belongs to the smallest x (5 mi/wk). Cadets who carry their Q16 table over in x-order will place the points left-to-right backwards.
Linearity — violated. The residuals form a clear U-shape: positive at both ends, negative through the middle, not randomly scattered about e = 0. The line systematically over-predicts in the middle and under-predicts at both ends; a predictable error is a modeling failure, not noise. Next: fit a nonlinear trendline — a quadratic for a single bend — and compare on the test set, not on R2.
Constant variance — no evidence of a violation. Residual magnitudes are 6, 3, 3, 3, 3, 6; the vertical spread neither fans out nor compresses systematically as ŷ increases. With six points this is a weak read, and cadets should say so.
Outliers — none. No residual sits far outside the range of the others; the two +6 values are the extremes but are equal in size and part of the pattern, not isolated from it.
Compute the mean, standard deviation, minimum, and maximum of the six training residuals.
The mean of the residuals should be very close to zero for a correctly fitted least-squares model. Is yours? What would it mean if the mean were substantially different from zero?
Mean = 0.00 min SD = 4.65 min Min = −3 min Max = +6 min
Work for the SD: the residual mean is 0, so Σe2 = 36+9+9+9+9+36 = 108; s = √(108 ÷ 5) = √21.6 = 4.6476.
The mean is exactly zero, as it must be for a least-squares fit — a property of the method, so it confirms your arithmetic but says nothing about whether the model is any good. A mean substantially different from zero would mean the line was not the least-squares line: an arithmetic error, a model fit on different data than it is scored on, or a trendline forced through the origin.
Three cadets were held out of the fit entirely. Complete the table using ŷ = 210 − 1.2x.
| x (mi/wk) | y actual | ŷ predicted | |y − ŷ| |
|---|---|---|---|
| 8 | 208 | ||
| 18 | 183 | ||
| 28 | 185 | ||
| Test MAE | |||
In two to three sentences, compare your test MAE against the training MAE you computed in Question 16(b) and state what the two numbers mean for using this model on this term's cadets.
| x | y | ŷ | |y − ŷ| |
|---|---|---|---|
| 8 | 208 | 200.4 | 7.6 |
| 18 | 183 | 188.4 | 5.4 |
| 28 | 185 | 176.4 | 8.6 |
| Test MAE = 21.6 ÷ 3 | 7.20 | ||
Test MAE = 7.20 minutes against a training MAE of 4.00 — the model is 80% worse on cadets it has never seen.
Model answer: "On the cadets used to build it, the model predicts ruck time to within about 4 minutes. On cadets held out of the fit it is off by about 7 minutes, so 7 minutes — not 4 — is the honest accuracy to plan around. Some gap is normal because the fit is tuned to the training cadets; a gap this large also matches the curved residual pattern, which says the straight line is the wrong shape. Usable for coarse triage, but I would not use it to separate a cadet predicted at 178 minutes from one at 183."
Lessons 19–21. Building a class label, standardizing with training statistics, distances, the majority vote, and the effect of k.
The commander now wants a simpler prediction: will a cadet earn a GO on the 12-mile march? KNN requires a categorical dependent variable, so the continuous ruck time is converted to a class at the Army standard of 180 minutes.
| Cadet | Weekly mileage (mi/wk) | 2-mile run (min) | Ruck time (min) | Class |
|---|---|---|---|---|
| A | 8 | 15 | 205 | NO-GO |
| B | 16 | 14 | 190 | NO-GO |
| C | 20 | 17 | 198 | NO-GO |
| D | 20 | 13 | 182 | NO-GO |
| E | 24 | 11 | 170 | GO |
| F | 32 | 14 | 176 | GO |
Cadet Q, whose class is unknown, trains 30 mi/wk and runs 2 miles in 12.0 minutes.
(a) The threshold is 180 minutes, the Army standard for the 12-mile foot march: ruck time ≤ 180 is a GO. It is defensible because it is the decision boundary that already exists in the real problem — the commander's question is literally "who will pass," so the label matches the decision. Splitting at the median instead would force exactly half the company into each class by construction, producing a label that is an artifact of this sample rather than a fact about the standard, and the model would answer a question nobody asked.
(b) The training mean and standard deviation define the scale the model was built on, so test records must be mapped onto that same scale to be comparable; re-standardizing with test statistics moves the yardstick between fitting and scoring, and the distances stop meaning the same thing. It also leaks information from the test set into the modeling process, destroying the test set's one job — being an honest stand-in for data the model has never seen. In deployment you would have no test set to compute statistics from anyway: a single new cadet has no mean.
(a) Sum = 8+16+20+20+24+32 = 120 → x̄ = 20 mi/wk
Deviations −12, −4, 0, 0, +4, +12; squares 144, 16, 0, 0, 16, 144; sum = 320 → s = √(320 ÷ 5) = √64 = 8 mi/wk
(b) z1* = (30 − 20) ÷ 8 = 1.25 z2* = (12 − 14) ÷ 2 = −1.00
Read Q off those numbers: trains 1.25 SD more than typical, runs 1.00 SD faster than typical.
Complete the table, then show one full distance calculation.
| Cadet | z1 | z2 | d from Q | Rank | Class |
|---|---|---|---|---|---|
| A | −1.50 | 0.50 | NO-GO | ||
| B | NO-GO | ||||
| C | NO-GO | ||||
| D | NO-GO | ||||
| E | GO | ||||
| F | GO |
| Cadet | z1 | z2 | Δz1 | Δz2 | d | Rank | Class |
|---|---|---|---|---|---|---|---|
| A | −1.50 | 0.50 | −2.75 | 1.50 | 3.13 | 6 | NO-GO |
| B | −0.50 | 0.00 | −1.75 | 1.00 | 2.02 | 4 | NO-GO |
| C | 0.00 | 1.50 | −1.25 | 2.50 | 2.80 | 5 | NO-GO |
| D | 0.00 | −0.50 | −1.25 | 0.50 | 1.35 | 3 | NO-GO |
| E | 0.50 | −1.50 | −0.75 | −0.50 | 0.90 | 1 | GO |
| F | 1.50 | 0.00 | 0.25 | 1.00 | 1.03 | 2 | GO |
Sample calculation, nearest cadet E: d = √[(0.50 − 1.25)2 + (−1.50 − (−1.00))2] = √(0.5625 + 0.25) = √0.8125 = 0.90
Neighbor order: E, F, D, B, C, A.
(a) Apply the rule of thumb k ≈ √ntrain, rounded to the nearest odd integer.
ntrain = √ntrain ≈ initial k =
Why do we prefer an odd value of k for a two-class problem?
(b) Classify cadet Q by majority vote at each k, naming the neighbors used.
| k | Neighbors used (in order) | Vote | Predicted class |
|---|---|---|---|
| 1 | |||
| 3 | |||
| 5 |
(a) ntrain = 6; √6 ≈ 2.45; nearest odd integer = k = 3
Why odd: with two classes an even k can split the vote evenly (2–2, 3–3), leaving the classifier with no majority and no principled tiebreak. An odd k guarantees a strict majority.
| k | Neighbors | Vote | Predicted |
|---|---|---|---|
| 1 | E | 1 GO – 0 NO-GO | GO |
| 3 | E, F, D | 2 GO – 1 NO-GO | GO |
| 5 | E, F, D, B, C | 2 GO – 3 NO-GO | NO-GO |
The prediction flips between k = 3 and k = 5. That instability is itself a finding and sets up Question 24.
Four held-out cadets were classified at each k. Carry your three predictions for cadet Q down from Question 23, mark them correct or incorrect, then fill in the three accuracies.
| Test cadet | mi/wk | 2-mi run | True class | k=1 | k=3 | k=5 |
|---|---|---|---|---|---|---|
| Q | 30 | 12.0 | GO | Q23 | Q23 | Q23 |
| T2 | 12 | 17.0 | NO-GO | NO-GO ✓ | NO-GO ✓ | NO-GO ✓ |
| T3 | 28 | 13.0 | GO | GO ✓ | GO ✓ | NO-GO ✗ |
| T4 | 32 | 16.0 | NO-GO | GO ✗ | NO-GO ✓ | NO-GO ✓ |
| Test accuracy | ||||||
Which k would you use, and why? Explain both what goes wrong at k = 1 and what goes wrong at k = 5.
Accuracy: k = 1 → 3/4 = 75% k = 3 → 4/4 = 100% k = 5 → 2/4 = 50%
Use k = 3. It has the highest test accuracy, and it is also what the √ntrain rule produced independently — the rule and the evidence agree.
What goes wrong at k = 1: the prediction rests entirely on one neighbor, so a single atypical training record decides the answer. Cadet T4 is the demonstration — its nearest neighbor is F, a GO, so k = 1 calls it GO, but T4 is really a NO-GO. This is overfitting: k = 1 reproduces the training data perfectly (each point is its own nearest neighbor, so training error is always 0) while generalizing worse.
What goes wrong at k = 5: the neighborhood grows so large it stops being local — it reaches across the whole feature space and washes out the structure near the target. Worse, the training set holds only 2 GO cadets out of 6, so at k = 5 a GO majority is arithmetically impossible: at most 2 of any 5 neighbors can be GO. The model can no longer predict the minority class at all, which is why it misses both true GO cadets.
Extending that: at k = 6 = n every cadet gets the same answer — NO-GO, the majority class — regardless of their features. That is the classification analogue of the mean-baseline model in Question 25.
Lessons 8, 16, 19–21. Comparing models on the right metric and knowing what each family actually assumes.
Complete the summary table. Write N/A where a metric does not apply, and be prepared to say why.
| Model | R2 | Train MAE | Test MAE | Test accuracy |
|---|---|---|---|---|
| Baseline — predict the training mean (189 min) | N/A | 9.00 | 9.67 | |
| Linear regression, ŷ = 210 − 1.2x | 0.854 | Q16 | Q19 | |
| Degree-5 polynomial through all six training points | 1.000 | 0.00 | 14.50 | |
| KNN classification at your chosen k | Q24 |
Which model do you recommend, and why? Address the degree-5 polynomial specifically.
| Model | R2 | Train MAE | Test MAE | Test accuracy |
|---|---|---|---|---|
| Baseline (mean, 189 min) | N/A | 9.00 | 9.67 | N/A |
| Linear regression | 0.854 | 4.00 | 7.20 | N/A |
| Degree-5 polynomial | 1.000 | 0.00 | 14.50 | N/A |
| KNN classification, k = 3 | N/A | N/A | N/A | 100% |
Why the N/A cells: R2 and MAE measure error in the units of a continuous response; KNN here predicts a category, and there is no meaningful distance between GO and NO-GO to average. Conversely the three regression models produce a number of minutes, not a class, so accuracy does not apply unless you first threshold their predictions at 180.
Recommendation. Among the models predicting ruck time, linear regression: selection is made on test MAE, the only column estimating performance on unseen cadets, and 7.20 beats the do-nothing baseline of 9.67. The degree-5 polynomial is the trap — R2 = 1.000 and training MAE = 0.00 look perfect because a degree-5 curve has exactly enough flexibility to pass through six points, fitting the noise along with the signal. Its test MAE of 14.50 is the worst of the three, worse than predicting the mean. That is overfitting, and it is the direct answer to HW2's question about whether a model can have a higher R2 and a worse test error: yes, and this is how.
A cadet who argues for KNN instead is also correct if they note it answers the commander's actual question — who passes — rather than predicting a time and thresholding afterward. Award full credit for that reasoning.
In two to three sentences, describe one fundamental difference between the way linear regression makes a prediction and the way KNN makes a prediction. What does each assume, or not assume, about the relationship between the inputs and the output?
Linear regression is a global, parametric model. It commits in advance to a shape — a straight line — and compresses the entire training set into two numbers, the slope and the intercept. Once fitted, the training data is discarded; a prediction is just an arithmetic evaluation of the equation. It assumes the relationship really is linear across the range, and when that fails you see it in the residual plot.
KNN is a local, non-parametric model. It assumes no functional form at all. It keeps every training record and, at prediction time, finds the records nearest the target and lets them vote. Its only assumption is that nearby points behave similarly — which is why the distance metric, and therefore standardization, does all the work.
The trade: regression extrapolates (with the risks of Q15b) and gives an interpretable slope; KNN adapts to any shape but cannot extrapolate beyond its neighbors, has no coefficient to interpret, and degrades when features are on mismatched scales or the training set is small.