Descriptive Statistics

6 concepts

In this chapter

A list of numbers is not yet a description of anything. This chapter turns raw data into a small set of pictures and numbers that say what it actually looks like: its shape, its center, and its spread. Each tool builds on the last, from through to the that closes the chapter.

Reading a Histogram: Where the Data Piles Up

Definition

A histogram displays data by grouping values into equal-width intervals, or bins, and drawing a bar over each whose height is the count, or proportion, of values it contains.

The test

Check two things: the bars touch, since the bins share boundaries along one continuous scale; and bar height encodes frequency, never the raw values themselves. A stem-and-leaf plot, line graph, or bar graph can also display data, but only a histogram bins a wide-ranging quantitative variable this way.

Example

A bootcamp records completion time, in hours, for 60 hypothetical learners. Sorted into equal-width bins, most bars cluster mid-range, with one or two shorter bars stretching out to one side — the shape a histogram exists to reveal at a glance, before any single number is calculated.

The trap

"A histogram is just a bar chart, so the bars can be reordered however looks best." A bar chart's categories carry no inherent order and its bars are drawn separated; a histogram's bins are numeric, fixed in sequence, and touching, because order is itself part of what the picture shows. "The tallest bar shows the single most common value." It shows the most populated bin, which can hold many different values — a histogram trades individual values for a view of the data's overall shape.

Percentiles and Quartiles: Locating a Value in the Data

Definition

The p-th percentile is the value below which p percent of the data falls. Quartiles are the three percentiles that cut ordered data into four equal-sized groups: $Q_1$ (25th percentile), the median (50th), and $Q_3$ (75th).

The test

Sort the data first — percentile rank is meaningless on an unsorted list. To place a value, ask what share of the sorted data sits at or below it, not its raw size. The interquartile range, $IQR = Q_3 - Q_1$ , measures the middle half's width, and a value more than $1.5 \times IQR$ below $Q_1$ or above $Q_3$ is flagged as a likely outlier.

Example

A hypothetical placement test reports scores by percentile. A score at the 40th percentile means 40% of test-takers scored at or below it — most of the field scored higher, even though 40 sounds middling out of 100.

The trap

"Scoring in the 90th percentile means answering 90% of the questions correctly." Percentile rank compares you to other test-takers, not to a perfect score — a 90th-percentile result could be a raw 65% if everyone else scored lower still. " $Q_1$ and $Q_3$ always sit the same distance from the median." Only for symmetric data: quartiles are defined by rank, not distance, so lopsided data routinely stretches further on one side.

Expanded

When your calculator disagrees. Several standard methods exist for computing quartiles — statistical software implements nine of them, catalogued after Hyndman and Fan (1996). This book takes the median of each half, which gives $Q_1 = 16.5$ and $Q_3 = 30.5$ for the nine review times used throughout this chapter. R's default method interpolates between neighbouring values instead, and returns 18 and 28 on exactly the same data. Neither result is an error, and the difference is not in the data — it is in the definition. So when a tool disagrees with a worked example here, check which quartile method it uses before hunting for a mistake in the arithmetic.

The Box Plot: Five Numbers, One Picture

Definition

A box plot (box-and-whisker plot) draws a dataset's five-number summary — minimum, $Q_1$ , median, $Q_3$ , maximum — in one picture: a box from $Q_1$ to $Q_3$ with a line at the median, and whiskers to the minimum and maximum. John Tukey popularized the form in a 1977 book.

The test

The box's width is the , holding the middle half of the data; it shows where that half starts and ends, not how many points sit inside. A median line off-center in the box, or one whisker much longer than the other, signals a lopsided distribution before any arithmetic.

Example

A hypothetical tutoring service samples 9 students and logs minutes spent reviewing flashcards before a quiz: 12, 15, 18, 20, 22, 25, 28, 33, and 60 — one reviewed far longer than the rest. The worked construction below builds its box plot.

Worked example

Sort the data. 12, 15, 18, 20, 22, 25, 28, 33, 60 (already sorted; $n = 9$ ).
Find the median. $n$ is odd, so the median is the middle (5th) value: 22.
Find $Q_1$ . The median of the lower four values (12, 15, 18, 20): $(15+18)/2 = 16.5$ .
Find $Q_3$ . The median of the upper four values (25, 28, 33, 60): $(28+33)/2 = 30.5$ .
State the five-number summary. Min $=12$ , $Q_1=16.5$ , median $=22$ , $Q_3=30.5$ , max $=60$ ; draw the box from 16.5 to 30.5, the median line at 22, and whiskers out to 12 and 60.

Common slips: forgetting to re-sort before splitting the halves, and including the median itself in both halves when $n$ is odd, which shifts $Q_1$ and $Q_3$ .

The five-number summary — minimum, Q1, median, Q3, maximum — for the same nine-student dataset used in the worked construction above, drawn as a single box plot.drawn by figures/fig-boxplot-anatomy.py

The trap

"A wider box means more data is packed inside it." Unlike a histogram: box width shows how spread out the middle half is, not how many points are there — a box plot carries no frequency information at all. "The line inside the box is the mean." It is the median; the mean is not marked on a standard box plot and, when data is skewed, can sit well outside the box.

Expanded

Why software draws this differently. By the $1.5 \times IQR$ rule from the previous section, the upper fence for these nine times sits at $30.5 + 21 = 51.5$ — so the 60 is flagged as a likely outlier by this chapter's own rule. The box plot above still runs a whisker out to it, because the five-number summary defines the whiskers as reaching the minimum and maximum. NIST calls a fence-based box plot a useful variation: its inner fences are numerical cutoffs at $Q_1 - 1.5 \times IQR$ and $Q_3 + 1.5 \times IQR$ ; the whiskers instead end at the farthest observed values still inside those cutoffs. Base R's boxplot() (range = 1.5) and Matplotlib 3.11.1's pyplot.boxplot() (whis = 1.5) default to this Tukey-style rule and plot values beyond the whiskers as outliers. For these nine values, both calculate $Q_1=18$ and $Q_3=28$ , putting the upper cutoff at 43; their upper whisker ends at 33 and they plot 60 separately. Those are their own quartile calculations, not this chapter's hand quartiles of 16.5 and 30.5. The same data therefore produce two different pictures, and both are correct — the whisker convention is a choice, not a fact about the data.

Mean, Median, and Mode: Three Kinds of Center

Definition

The mean — $\bar{x}$ for a sample, or $\mu$ for a population — adds every value and divides by the count. The median is the sorted data's middle value, or the average of its two middle values when the count is even. The mode is the most frequent value; a dataset can have none, one, or several.

The test

The mean uses every value's size, so an extreme value pulls it; the median uses only rank and barely moves. When no value repeats, the dataset has no mode — a valid description, not a gap.

Example

The same 9 students from the above spent 12, 15, 18, 20, 22, 25, 28, 33, and 60 minutes. The mean is $\bar{x} = 233/9 \approx 25.9$ ; the median, the 5th sorted value, is 22. All nine minutes are distinct, so there's no mode.

The trap

"The mean is always the best measure of center, so it's always the right one to report." Here the mean, 25.9, exceeds all but one value, pulled up by that one student's 60 minutes, while the median, 22, better represents a typical student — neither is wrong, but picking whichever looks better is dishonest. "Every dataset has exactly one mode." A dataset can have none, as here, one, or several equally common values.

Skewness: When the Tail Pulls the Mean Away

Definition

A distribution is skewed when one tail stretches further than the other, and symmetric when neither does. Skew pulls the toward the long tail while the median resists it, so the gap between them reads out the skew.

The test

Find the longer tail first, then name the skew after it: right-skewed if it stretches right, left-skewed if left. An off-center median in a is often the first visible sign. Right-skewed data typically has mean greater than median, and left-skewed data typically the reverse — a generalization, OpenStax notes, that can fail on some discrete datasets.

Example

Back to the 9 students: the , 25.9, sits above the median, 22, pulled up by the one 60-minute outlier — a right-skewed , long tail right, most students bunched low.

Three simulated distributions — left-skewed, symmetric, and right-skewed — each with its mean (dashed red) and median (dotted blue) marked, showing how skew separates the two.drawn by figures/fig-skewness-shapes.pyThis figure becomes interactive in the app.

The trap

"Right-skewed means most of the data is on the right." The opposite: the tail, not the bulk, is on the right; most values, and the mode, sit on the low side where the bars are tallest. "The mean is still the most representative measure even when data is skewed." Skew is exactly what pulls the mean from where most data sits — resisting that pull is the median's job.

Standard Deviation and Variance: Measuring the Spread

Definition

The standard deviation measures the typical distance between a data value and the , in the data's own units; the variance is the same idea in squared units, before the square root is taken. A sample uses $s$ , dividing by $n-1$ ; a population uses $\sigma$ , dividing by $N$ .

The test

Ask what a "typical" distance from the mean looks like, not just the distance between the two extremes — standard deviation uses every value, so one distant point moves it, but far less than it moves the range. Two datasets can share a mean and a range and still have very different standard deviations.

Example

The same 9 students' minutes give $\bar{x} \approx 25.9$ . The worked computation below finds the sample standard deviation $s$ .

Worked example

Find the mean. $\bar{x} = 233/9 \approx 25.9$ .
Find each deviation and square it. For example, $12-25.9=-13.9$ , squared $\approx 193$ ; repeat for all nine values.
Sum the squared deviations. $\displaystyle \sum(x-\bar{x})^2 \approx 1{,}642.9$ .
Divide by $n-1$ . $s^2 = 1{,}642.9 / 8 \approx 205.4$ , the sample variance.
Take the square root. $s = \sqrt{205.4} \approx 14.3$ minutes.

Common slip: dividing by $n$ instead of $n-1$ for a sample, which understates $s$ .

The trap

"A bigger standard deviation just means the numbers themselves are bigger." Scale and spread are independent: exam scores in the 90s can vary more, point to point, than ages in the 20s. "Standard deviation and range measure basically the same thing." The range uses only the two extremes; standard deviation uses every value, so one stray point moves the range a great deal and the standard deviation only slightly.

Two simulated datasets sharing the same mean but different standard deviations, plotted side by side, so greater spread is visible directly and not just a bigger number in a formula.drawn by figures/fig-spread-comparison.py

Describing One Dataset, Start to FinishSynthesis

Put the nine students' flashcard minutes through every tool in this chapter and they agree on one story. A of many such samples would show a tall cluster on the low end and a thinning tail toward the high one. The , 16.5 and 30.5, box the middle half; the built from them shows a median line sitting closer to the box's low side than its high one — a visual tell of skew before any arithmetic. The , 25.9 and 22, disagree by close to four minutes, and that gap is made numeric: the one 60-minute outlier pulls the mean toward the tail while the median holds its ground. The , about 14.3 minutes, says how far a typical student's time sits from that pulled-up mean — a number that would mean little without the shape, center, and quartiles already in hand. No picture here replaces the numbers, and no number replaces the picture; a dataset is fully described only when both agree.

Sources

1 OpenStax Introductory Statistics §2.1: Stem-and-Leaf Graphs (Stemplots), Line Graphs, and Bar Graphs — source
2 OpenStax Introductory Statistics §2.2: Histograms, Frequency Polygons, and Time Series Graphs — source
3 OpenStax Introductory Statistics §2.3: Measures of the Location of the Data — source
4 OpenStax Introductory Statistics §2.4: Box Plots — source
5 OpenStax Introductory Statistics §2.5: Measures of the Center of the Data — source
6 OpenStax Introductory Statistics §2.6: Skewness and the Mean, Median, and Mode — source
7 OpenStax Introductory Statistics §2.7: Measures of the Spread of the Data — source
8 Wickham & Stryjewski, "40 Years of Boxplots" (on John Tukey's 1977 introduction of the box plot) — source
9 R stats::quantile documentation — nine quantile types after Hyndman & Fan (1996); type 7 is R's default — source
10 NIST/SEMATECH e-Handbook of Statistical Methods §1.3.3.7: Box Plot (the fence-based box plot is a variation; whiskers end at the farthest observations inside the inner fences) — source
11 R graphics::boxplot documentation (default range = 1.5; whiskers end at the most extreme data point within range and outliers are drawn as points) — source
12 Matplotlib 3.11.1 pyplot.boxplot documentation (default whis = 1.5; whiskers end at the farthest data within the cutoff and values beyond are plotted as outliers) — source