1. ## Some Interaction Information Examples

Interaction information between three random variables has a much less immediate interpretation compared to other information quantities. This makes it more tricky to work with. An i.i. term $I(X;Y;Z)$ between $X,\ Y$ and $Z$, could be either negative:

If we are trying to specify $X$ with …

2. ## Porting PostmarketOS to the Motorola Photon Q

PostmarketOS has a goal to make old cell phones useful past their shelf life. Right now their efforts are towards getting Alpine Linux to run on as many phones as possible. The majority their codebase right now is a tool called pmbootstrap that automates parts of porting Alpine. Porting is …

3. ## Point KL-Divergence is not Very Negative Very Often

If $X\sim P$ then for any distribution $Q$ it is unlikely that $Q$ ascribes much greater density to $X$'s outcome than $P$ does. In fact if $P,Q$ have PDFs $f_P, f_Q$, then:

\begin{align} \mathbb{P}(f_P(X)\leq c f_Q(X)) &= \int \mathbf{1}_{\{x …
4. ## Tails of Probability Distributions are Nowhere Dense

Lately I have been thinking about Kullback-Liebler divergence on probability distributions, also called the KL discriminant or relative entropy. It is often useful to think about this quantity as a distance, even though it doesn't behave much like one: it's not symmetric, it doesn't follow the triangle inequality, and even …

5. ## Thoughts on Demosaicing for X-Trans Sensors

A lot of new Fujifilm cameras use their own brand of 'X-Trans' sensors which have a non-Bayer color mosaic. Fujifilm claims that the less regular arrangement makes it more uncommon for edges in the scene to make moire patterns with the mosaic. They say this justifies not including an optical …

6. ## Update: Extracting Information from Noisy Observations

This is meant as a follow-up for the last post's motivation. In summary we have a bipartite graph $G=(U,V,E)$ with bi-adjacency matrix $A$ and with $|U|\approx rk,$ $|V| \approx ck,$ $|E|\approx rck,$ $\mathsf{deg}(u)\approx c,$ $\mathsf{deg}(v)\approx r,$ and a sequence …

7. ## Thoughts about Extracting Information from Sequences of Noisy Observations

Often we have some quantity of interest that we can only observe indirectly. In these situations, we would like to know how much relevant information we can extract from our noisy, indirect observations, and how to perform the extraction. Figuring out how to do this would in theory allow you …