The Central Limit Theorem is one of Statistics and Data Science’s most significant and beautiful theorems. This blog will provide an example of how to visualize CLT with Python. Let’s begin with the basics. The distribution of means vs the distribution of scores When talking about statistics most people focus on the distribution of scores.
That is, if we have 6, 34, 11, 98, 52, 34, 13, 4, 48, 68, we can calculate the mean of the 10 scores: and subsequently, the population standard deviation is: The distribution of means, on the other hand, looks at groups of 2 or 3 or more scores in a sample. For example, if we group the scores out of convenience, we can have 6 and 34 as group 1, 11 and 98 as group 2, and so on. We look at the mean of all the groups, which are (6+34)/2=10, (11+98)/2 = 54.5, etc.
And then, we calculate the mean of the distribution of 5 means, which should be the same as the mean of the distribution of scores 36.8. Therefore, you can imagine the distribution of scores as studying 6, 34, 11, 98, 52, 34, 13, 4, 48, 68 (the 10 scores) and the distribution of means as studying 10, 54.5, 43, 8.5, 58 (the 5 means).
This is just a demonstration. In reality, we bootstrap (random sampling with replacement) from the 10 scores and have an infinite number of groups of the same size N. Next, let’s dive into a simple coin example. CLT explained with an example.
Let’s say we have 1000 coins. We denote the head and tail respectively as 1.
I emphasize ideally because in reality, one might easily end up with a 501/499 or 490/510 split. So why is 500/500 ideal?
This intuitively seems simple. This is how mathematically we can prove it. We need to discuss probability in order to answer this question. Statistics is the study of the frequency and likelihood of past events. Probability involves predicting future events. Before this, statistics was all we were talking about.
We looked at experiments and counted heads and tails. We can predict the probability of something happening when we have a large sample. The Law of Large Numbers predicts what is likely to occur. LLG requires that the sample must be large enough to allow for infinite sampling. Let’s take an example: let us do another experiment using fair coins but with 100000000000 more.
This is because the chance of tossing either a head nor a tail on a coin that’s fair is equal, which in fact are both 0.5. If the sample is larger, we expect that the final outcome will be close to half-way.
Importantly, increasing the sample size doesn’t change the distribution of scores. The outcome of a fair coin toss will not change because CLT refers only to the distribution of the means and not the distributions of the scores. The distribution of infinite scores allows us to easily determine the standard deviation and the mean. We will get n/2 heads and 2/2 tails when we throw n fair coins (n is infinite). We now look at CLT Visualized.
This means that 2 coins can be grouped together to create infinite numbers of two coins. There are three possible ways to make 2 coins: 0, 0.5 and 1.
. The chance of the first and second coins being tails is half 1/2 = 1/4. The same goes for the three other coins.
The probability of getting a mean value of 0. (both tails) equals 1/4. A mean of 0.50 (one tail, one head) would be 1/4*2= 1/2. A mean of 1 for both heads is possible, but it’s also very unlikely. This is the distribution.
The y-axis now represents Probability (aka count) but it is still the same shape, regardless of which y axis is used. We now have three bars that are topped by a peak, as opposed to the bar chart before.
We calculate the standard deviation and mean again. The sd now is lower, while the mean remains the same. We can also group three coins together. There are three possible ways to group 3 coins: 0, 1/3, 2/3 and 1. Total, there are 8 possible combinations of 2 *2 *2.
The probability of reaching a mean value of 0 in a given case is p(0) = 1/8. Similarly, p(0. 33) = p(0. 67) = 3/8 andp(1) = 1/8.
The new distribution has four bars. This is 1 more than in the previous one. While the mean remains the same, the sd decreases. This seems like a consistent pattern. How would the distribution, mean and standard deviation change if N was increased? It is possible to see the pattern. I visualized and calculated more results. Below is N, which indicates the number of coins found in each group. According to the distribution of scores, N is 1. Photo by author. It is obvious that N gets larger, and the distribution of means becomes closer to normal distribution. The bars along the sides are hard to see when N is very large. However, the probability of finding the right means ranges from 0-1.
You can see also that N is always very large, so you cannot see the bars on the sides. However, the probabilities are too small to be able to tell the difference between the possible means. The Standard Error As N grows, we can quantify changes in standard deviation.
Again, we can see that the mean stays the same and the sd of the distribution of means drops as N gets larger. The standard deviation is the measurement of how many means are distributed.
It is also known as the standard error or SE. It becomes a distribution or means when N is greater than 2 As N grows, SE’s denominator increases and SE falls. We have now concluded our discussion by looking at the coin illustration closely. The following are the points we learned:
