Random Notes on Randomness
random learning notes. (I studied probability, statistics, game theory, back in ~1993. Here's a refresher for myself.)
[ Randomness ] [ https://en.wikipedia.org/wiki/Randomness ]
[ Random variable ] [ https://en.wikipedia.org/wiki/Random_variable ]. → A variable with random values. aka stochastic variable.
[ Random process ] [ https://en.wikipedia.org/wiki/Random_process ] aka “stochastic process” is a collection of random variables.
random variable can be discrete (such as any integer from 1 to 10), or continuous (such as any real number between 1 to 10)
probality function → a function that takes a value (of possible values of a random variable) and returns its probability.
- a probability function for discrete random variable is called probability mass function
- a probability function for continuous random variable is called probability density function
probability is a number between 0 and 1, inclusive. 0 means never gonna happen. 1 means sure thing.
[ Probability distribution ] [ https://en.wikipedia.org/wiki/Probability_distribution ] → the possible values of a random variable and their associated probabilities.
a probability distribution can be for one single random variable (this is most common), or multiple random variables. When it's single variable, it's called univariate. For multiple variable, it's multivariate.
[ Monte Carlo method ] [ https://en.wikipedia.org/wiki/Monte_Carlo_method ] basically, a method to test some theory or answer some question by simulating the event using random data. « Monte Carlo methods (or Monte Carlo experiments) are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results; i.e., by running simulations many times over in order to calculate those same probabilities heuristically just like actually playing and recording your results in a real casino situation: hence the name.»
Monte Carlo methods vary, but tend to follow a particular pattern:
- Define a domain of possible inputs.
- Generate inputs randomly from a probability distribution over the domain.
- Perform a deterministic computation on the inputs.
- Aggregate the results.
For example, consider a circle inscribed in a unit square. Given that the circle and the square have a ratio of areas that is π/4, the value of π can be approximated using a Monte Carlo method:
- Draw a square on the ground, then inscribe a circle within it.
- Uniformly scatter some objects of uniform size (grains of rice or sand) over the square.
- Count the number of objects inside the circle and the total number of objects.
- The ratio of the two counts is an estimate of the ratio of the two areas, which is π/4. Multiply the result by 4 to estimate π.
Entrophy
[ Entropy (information theory) ] [ https://en.wikipedia.org/wiki/Entropy_%28information_theory%29 ]
PARADOXES OF RANDOMNESS By Gregory Chaitin. (Complexity, Vol 7, No 5, May/June 2002, Pp 14-21) At http://www.cs.auckland.ac.nz/~chaitin/summer.html