Questions tagged [entropy]
Entropy is a measure of the uncertainty in a random variable.
602
questions
2
votes
1
answer
52
views
I can't work out the entropy formula for TraMineR
I used the ggseqplot() library to calculate the entropy from the TraMineR library. To familiarise myself with the formula, I tried unsuccessfully to calculate it by hand using simple example data.
...
0
votes
0
answers
16
views
Is there an error with my sample entropy estimator for 3D multivariate signal?
I wrote a Matlab function to calculate sample entropy of a multivariate time signal (X,Y,Z positions) similarly to the approximateEntropy built-in function from Matlab. I tried to check my answers ...
1
vote
1
answer
62
views
Use "/dev/urandom" with GMP library to generate a very big random number
I'm using Debian Bookworm and GMP library version 60201.
I need to generate a random number with GMP library but I have 2 requirements, if possible, of course!
1 - Generate a number between 0 and a ...
0
votes
1
answer
29
views
Can the total Entropy of all clusters be greater than 1, after classification?
After doing k-means classification on a dataset (value of k = 3), I tried to find out the total entropy of all the clusters. (Total number of datapoints, or, the total length of the dataset was : 500)
...
0
votes
0
answers
17
views
How can I calculate in MATLAB the entropy and mutual information of points moving through 3D space?
I have 838 points from a 3D reconstruction of an object and I want to know the best way to calculate the entropy of each point and the mutual information between them. I have 3D coordinates for each ...
0
votes
0
answers
30
views
Calculating Chao-Shen corrected Jensen-Shannon Divergence
I am trying to calculate JS divergence of two distributions P and R with Chao-Shen correction (https://link.springer.com/article/10.1023/a:1026096204727). P and R are simple arrays where each element ...
0
votes
0
answers
24
views
KL divergence corrected for limited sample size bias
I have a reference distribution R from which I am sampling to create distributions of different sample sizes. These new distributions are of same dimensions but with different numbers of data points. ...
0
votes
0
answers
39
views
Error: Image dimensions and neighborhood dimensions do not match
When I tried to compile the following code, I encountered the error below, but I didn't understand what I should do. Do you have any idea, please?
Error:
Traceback (most recent call last):
File &...
0
votes
1
answer
50
views
Is it feasible to use weightthem - ebal (R statistics) on an interaction as a Treatment?
I´m trying to find out if there are differences in outcomes between students that were treated in a given school type vs. students that were treated in any other school type.
I used PSM to find ...
0
votes
0
answers
36
views
Calculating conditional entropy bigrams using NLTK and Kneser-Ney smoothing
I am trying to estimate the conditional entropy of a text source at the bigram level. In order to get a good estimate, I need estimates for the probabilities of bigrams. After doing some reading, it ...
0
votes
0
answers
47
views
Calculating relative entropy for a latent class analysis - two methods give different results
I'm using R poLCA to run a latent class model with 4 categorical indicators (3 levels, 3 levels, 9 levels and 5 levels). As poLCA doesn't compute relative entropy, I have found two formulas for ...
0
votes
0
answers
46
views
Estimating differential entropy with weighted samples?
I have an unknown continuous probability distribution p(x) that is expensive to sample from, but cheap to evaluate, and I would like to estimate its differential entropy. Some other details that might ...
0
votes
1
answer
57
views
Calculate Entropy of various text strings in an extra column od a table in Rstudio
I am currently desperate regarding my master thesis in which I am working with R. I hope someone can help me!
I have a dataframe with about 70,000 rows and 38 columns. Now I want to calculate the ...
0
votes
1
answer
123
views
Calculation of compression ratio using arithmetic encoding?
Arithmetic encoding is one of the most famous entropy encoding techniques, and I am using it to encode an image. For this, I am using the built-in function of Matlab that also gives other values such ...
0
votes
0
answers
30
views
Information content of a (1D) curve (i.e. spectroscopy)
I am looking for a measure to quantify the information content of a 1D curve. To explain, here is an example in python:
import numpy as np
import matplotlib.pyplot as plt
x, dx = np.linspace(0, 10, ...
0
votes
0
answers
17
views
What statistical characteristics does each image have? (feat. noise, image entropy)
I am a student studying image processing. I'm still lacking a lot, so I'm leaving a question here.
Image Noise
1-1) Is the statistical distribution of noise for each image unique?
For example, I ...
0
votes
1
answer
69
views
How many bins are applied to an image of different types in skimage.measure.shannon_entropy()
How does the skimage.measure.shannon_entropy() compute the histogram before returning the entropy result? I pass images in the format of 1,2 and 4 byte signed and unsigned grayscale images and also ...
0
votes
1
answer
353
views
Cryptographically secure random number generator for software entropy in Node.js
I'm trying to generate key pairs (public and private) and I'm looking for cryptographically secure libraries that allow me to generate good entropy (both hardware and software) in a node.js ...
1
vote
1
answer
361
views
Algorithm to randomize output order of a short array in Go
The key difference between this question and the lots of duplicated answers is that, the input array is short, only 3 elements. --
Say I have an ordered set of int. The size of array is only 3 (or a ...
0
votes
0
answers
66
views
How is PNG compressing more than the entropy limit?
I have an image of dimension 2868x4312, whose shannon-entropy is ~6.46. It implies the image can't be compressed beyond ~9.52 MB (2868x4312x6.46/8/1024/1024). But when I save this image as PNG using ...
0
votes
0
answers
78
views
Increase entropy pool by writing to /dev/random (macOS)?
I know there's rng-tools for Linux, but haven't found a port for macOS, so I'm trying to do it manually. I understand you can write to /dev/random to fill the entropy pool (at least in Linux, probably ...
0
votes
2
answers
25
views
entropy-like measure to quantify the robustness of random forest prediction, using fraction of probability
Imagine I have performed a random forest to classify according to three classes: A, B, and C.
My correct get for a given sample is [1, 0, 0], because it belongs to class 1.
I have two different models ...
0
votes
0
answers
38
views
Cross entropy value higher than entropy
I'm getting a result for entropy that is higher than for cross-entropy, which shouldn't be possible:
import numpy as np
# Define some sample data
y_true = np.array([0.10, 0.40, 0.50])
y_pred = np....
0
votes
0
answers
69
views
How to adjust Arithmetic Coding parameters in integer form?
Problem:
I'm using the arithmetic coding model based on this PDF paper, which is in integer form to realize an unbounded precision.
But the application has some problems: all we calculate is the ...
0
votes
0
answers
27
views
How to calculate matrix entropy?
I’m a French student that is taking a look at the matrix entropy ! Could you please explain me how do you manually calculate the entropy of a matrix ? My goal is to calculate it with a python programm....
0
votes
2
answers
86
views
Unable to find the cause of segmentation fault of huffman coding to do entropy encoding and decoding in c
I have been looking at the code to do entropy encoding and decoding with of huffman coding in c, and I cannot seem to find the reason why I keep getting segmentation fault.
#include <stdio.h>
#...
0
votes
0
answers
603
views
Spectral entropy of a signal in Python
I wish to find spectral entropy of below sinusoidal signal:
## Impprt the libraries
import numpy as np
import entropy as ent
Fs = 100 ## Sampling frequency (Hz)
f = 1 ##...
-1
votes
1
answer
118
views
what is the range of gini impurity when more than 2 classes?
when we are building a decisiontree, we are usually calculating the gini impurity at each node.
I am interested to see the range of gini impurity in case of more than 2 classes.
Because entropy always ...
0
votes
1
answer
174
views
How to calculate time-frequency domain renyi entropy?
This is the definition of Renyi entropy in the time-frequency domain. I will use this function to calculate the Renyi entropy of a spectrogram. px(t,f) will be the outcome of this spectrogram. For ...
0
votes
0
answers
61
views
I don’t get information gain equation in XGBoost
I’m currently studying XGBoost, and I learned that information gain in XGBoost is computed like this:
XGBoost information gain
What I’m curious is that, previously about information gain, I learned ...
0
votes
2
answers
313
views
what R Code to calculate the entropy for each level in a categorical variable
I have quite some categorical variable in my dataset, These variables have more than two levels each. Now i want an R code function (or loop) that can calculate the entropy and information gain for ...
0
votes
1
answer
94
views
PDF Entropy calculation
Last time mkl helped me a lot, hopefully he (or someone else) can help me with these questions too. Unfortunately I couldn't get access to the ISO norm (ISO 32000-1 or 32000-2).
Are these bytes used ...
0
votes
1
answer
232
views
Shannon entropy function returns -0.0 instead of 0.0
My Shannon entropy function for "aaaaa" should return 0.0 but gives me -0.0. How do I fix that?
1
vote
1
answer
54
views
Coverage-adjusted entropy estimation in Python
I need to estimate entropy for many variables in vocabulary data, and some of these have only small samples. I have previously done this using the Chao-Shen entropy estimation in R, but now I would ...
0
votes
1
answer
221
views
How to find Entropy or KL divergence between two data?
I have two datasets which are same shape: (576, 450, 5) where 576 is the number of examples, 450 is the time points and 5 is the number of channels.
I want to calculate entropy and KL-divergence ...
0
votes
1
answer
395
views
Total Gini impurity or entropy gain for a scikit-learn decision tree
How can I get the total weighted Gini impurity (or entropy) on a trained decision tree in scikit-learn? For instance, the following code on the titanic dataset,
import pandas as pd
import matplotlib....
0
votes
1
answer
23
views
Burp Sequencer, how to set significance level
I read this article How the randomness tests work and it appears the significance level for the analysis can be configured but I can't find any UI element or documentation to make that change.
3
votes
0
answers
99
views
Cannot identify mistake in computing Shannon entropy
According to this blog, Shannon entropy of the prime numbers approaches log_2(e) ≈ 1.442695 as the limit counting prime numbers approaches infinity. I wanted to see this for myself in Python using ...
0
votes
0
answers
158
views
Is there a better method for entropy calculation than shannon entropy for measuring entropy of a PowerShell script?
I thought it would be pretty easy to write a function which measures entropy in a string (or more likely copy / paste a method someone else made) in order to detect encoded PowerShell scripts. Googled ...
1
vote
0
answers
290
views
How to calculate the KL divergence for two multivariate pandas dataframes
I am training a Gaussian-Process model iteratively. In each iteration, a new sample is added to the training dataset (Pandas DataFrame), and the model is re-trained and evaluated. Each row of the ...
0
votes
1
answer
23
views
MSEn of data error Mobj unable to be identified
'def MSEn(dataset_mus, Mobj, Scales=3, Methodx='coarse', RadNew=0, Plotx=False):
class MSobject:
x=(5,2,[3.2])
Mobj = MSobject(x)
dataset_mus=np.squeeze(dataset_mus)
Func2 = globals()[Methodx.lower()]
...
1
vote
1
answer
212
views
Create relative entropy matrix from pandas dataframe
I have a dataframe of values, say:
df = pd.DataFrame(np.array([[0.2, 0.5, 0.3], [0.1, 0.2, 0.5], [0.4, 0.3, 0.3]]),
columns=['a', 'b', 'c'])
in which every row is a vector of ...
1
vote
2
answers
216
views
Contingency matrix to 1D format in Python
2x2 contingency matrix:
Cj
2 1
Ci
1 0
Translates to:
[[ 0 0 0 1 ]
[ 0 0 1 0 ]]
The contingency matrix represents the outcome of two clustering algorithms, each with two clusters. The ...
3
votes
2
answers
222
views
Efficient computation of entropy-like formula (sum(xlogx)) in Python
I'm looking for an efficient way to compute the entropy of vectors, without normalizing them and while ignoring any non-positive value.
Since the vectors aren't probability vectors, and shouldn't be ...
0
votes
1
answer
356
views
How many bits are required to encode information in probability set G = [0.001, 0.002, 0.003, 0.994]?
I am currently working on data compression and thought it would be a good time to read up on the basics of information theory to better understand data compression and its algorithms.
As I understand, ...
0
votes
1
answer
100
views
Adding and removing byte from a file in a loop
I'm a security researcher and am wondering how I could remove the added bytes from a file. This function below adds a byte every 4 bytes. But I'd like to know how I could remove those bytes as well. ...
1
vote
1
answer
150
views
Entropy calculation gives NaN - is applying na.omit a valid tweak?
By definition, the entropy is defined as:
entropy <- function (p) sum(-p * log(p))
I'm performing LCA using the poLCA package and trying to calculate entropy, which for some of my models are ...
1
vote
2
answers
3k
views
How to calculate mutual information in PyTorch (differentiable estimator)
I am training a model with pytorch, where I need to calculate the degree of dependence between two tensors (let's say they are the two tensors each containing values very close to zero or one, e.g. v1 ...
0
votes
0
answers
56
views
Extract possible sample combinations from multiple count constraints
I have some input data like this.
unique ID
Q1
Q2
Q3
1
1
1
2
2
1
1
2
3
1
0
3
4
2
0
1
5
3
1
2
6
4
1
3
And my target is to extract some data which satisfy the following conditions:
total count: ...
0
votes
0
answers
37
views
Is the runtime of df.groupby().transform() linear in the number of groups in the groupby object?
BACKGROUND
I am calculating racial segregation statistics between and within firms using the Theil Index. The data structure is a multi-indexed pandas dataframe. The calculation involves a lot of df....