What is the Average Value as the Number of Samples Increases

In the previous lesson, we reviewed the meaning of “expected value”: when you get a whole bunch of samples from a distribution, and a function on those samples, what is the average value of the function’s value as the number of samples gets large?

---

Revisiting the Naive Implementation of Expected Value

We gave a naive implementation:

public static double ExpectedValue<T>(
    this IDistribution<T> d,
    Func<T, double> f) =>
  d.Samples().Take(1000).Select(f).Average();

public static double ExpectedValue(
    this IDistribution<double> d) =>
  d.ExpectedValue(x=>x);

Issues with the Naive Approach

Though short and sweet, this implementation has some problems; the most obvious one is that hard-coded $1000$ in there; where did it come from? Nowhere, in particular, that’s where.

It seems highly likely that this is either too big, and we can compute a reasonable estimate in fewer samples, or that it is too small, and we’re missing some important values.

Let’s explore a scenario where the number of samples is too small. The scenario will be contrived but not entirely unrealistic.

Let’s suppose we have an investment strategy; we invest a certain amount of money with this strategy, and when the strategy is complete, we have either more or less money than we started with. To simplify things, let’s say that the “outcome” of this strategy is just a number between $0.0$ and $1.0$; $0.0$ indicates that the strategy has completely failed, resulting in a loss, and $1.0$ indicates that the strategy has completely succeeded, resulting in the maximum possible return.

Before we go on, we want to talk a bit about that “resulting in a loss” part. If you’re a normal, sensible investor and you have $\$100$ to invest, you buy a stock or a mutual fund for $\$100$ because you believe it will increase in value. If it goes up to $\$110$, you sell it and pocket the $\$10$ profit. If it goes down to $\$90$, you sell it and take the $\$10$ loss. But in no case do you ever lose more than the $\$100$ you invested. (Though of course, you do pay fees on each trade whether it goes up or down; let’s suppose those are negligible.) Our goal is to get that $10\%$ return on investment.

Now consider the following much riskier strategy for spending $\$100$ ...