Questions tagged [entropy]

Entropy is a measure of the uncertainty in a random variable.

Filter by
Sorted by
Tagged with
2 votes
1 answer
52 views

I can't work out the entropy formula for TraMineR

I used the ggseqplot() library to calculate the entropy from the TraMineR library. To familiarise myself with the formula, I tried unsuccessfully to calculate it by hand using simple example data. ...
JC_'s user avatar
  • 53
0 votes
0 answers
16 views

Is there an error with my sample entropy estimator for 3D multivariate signal?

I wrote a Matlab function to calculate sample entropy of a multivariate time signal (X,Y,Z positions) similarly to the approximateEntropy built-in function from Matlab. I tried to check my answers ...
Adam 's user avatar
1 vote
1 answer
62 views

Use "/dev/urandom" with GMP library to generate a very big random number

I'm using Debian Bookworm and GMP library version 60201. I need to generate a random number with GMP library but I have 2 requirements, if possible, of course! 1 - Generate a number between 0 and a ...
PsySc0rpi0n's user avatar
0 votes
1 answer
29 views

Can the total Entropy of all clusters be greater than 1, after classification?

After doing k-means classification on a dataset (value of k = 3), I tried to find out the total entropy of all the clusters. (Total number of datapoints, or, the total length of the dataset was : 500) ...
SSaha13's user avatar
0 votes
0 answers
17 views

How can I calculate in MATLAB the entropy and mutual information of points moving through 3D space?

I have 838 points from a 3D reconstruction of an object and I want to know the best way to calculate the entropy of each point and the mutual information between them. I have 3D coordinates for each ...
Adam 's user avatar
0 votes
0 answers
30 views

Calculating Chao-Shen corrected Jensen-Shannon Divergence

I am trying to calculate JS divergence of two distributions P and R with Chao-Shen correction (https://link.springer.com/article/10.1023/a:1026096204727). P and R are simple arrays where each element ...
ridul's user avatar
  • 1
0 votes
0 answers
24 views

KL divergence corrected for limited sample size bias

I have a reference distribution R from which I am sampling to create distributions of different sample sizes. These new distributions are of same dimensions but with different numbers of data points. ...
ridul's user avatar
  • 1
0 votes
0 answers
39 views

Error: Image dimensions and neighborhood dimensions do not match

When I tried to compile the following code, I encountered the error below, but I didn't understand what I should do. Do you have any idea, please? Error: Traceback (most recent call last): File &...
Made's user avatar
  • 101
0 votes
1 answer
50 views

Is it feasible to use weightthem - ebal (R statistics) on an interaction as a Treatment?

I´m trying to find out if there are differences in outcomes between students that were treated in a given school type vs. students that were treated in any other school type. I used PSM to find ...
Paul Fabian's user avatar
0 votes
0 answers
36 views

Calculating conditional entropy bigrams using NLTK and Kneser-Ney smoothing

I am trying to estimate the conditional entropy of a text source at the bigram level. In order to get a good estimate, I need estimates for the probabilities of bigrams. After doing some reading, it ...
chasmani's user avatar
  • 2,382
0 votes
0 answers
47 views

Calculating relative entropy for a latent class analysis - two methods give different results

I'm using R poLCA to run a latent class model with 4 categorical indicators (3 levels, 3 levels, 9 levels and 5 levels). As poLCA doesn't compute relative entropy, I have found two formulas for ...
Sointu's user avatar
  • 109
0 votes
0 answers
46 views

Estimating differential entropy with weighted samples?

I have an unknown continuous probability distribution p(x) that is expensive to sample from, but cheap to evaluate, and I would like to estimate its differential entropy. Some other details that might ...
LemonPi's user avatar
  • 1,046
0 votes
1 answer
57 views

Calculate Entropy of various text strings in an extra column od a table in Rstudio

I am currently desperate regarding my master thesis in which I am working with R. I hope someone can help me! I have a dataframe with about 70,000 rows and 38 columns. Now I want to calculate the ...
Sandra Roccuzzo's user avatar
0 votes
1 answer
123 views

Calculation of compression ratio using arithmetic encoding?

Arithmetic encoding is one of the most famous entropy encoding techniques, and I am using it to encode an image. For this, I am using the built-in function of Matlab that also gives other values such ...
Sahil Sharma's user avatar
0 votes
0 answers
30 views

Information content of a (1D) curve (i.e. spectroscopy)

I am looking for a measure to quantify the information content of a 1D curve. To explain, here is an example in python: import numpy as np import matplotlib.pyplot as plt x, dx = np.linspace(0, 10, ...
R. C.'s user avatar
  • 651
0 votes
0 answers
17 views

What statistical characteristics does each image have? (feat. noise, image entropy)

I am a student studying image processing. I'm still lacking a lot, so I'm leaving a question here. Image Noise 1-1) Is the statistical distribution of noise for each image unique? For example, I ...
함세준's user avatar
0 votes
1 answer
69 views

How many bins are applied to an image of different types in skimage.measure.shannon_entropy()

How does the skimage.measure.shannon_entropy() compute the histogram before returning the entropy result? I pass images in the format of 1,2 and 4 byte signed and unsigned grayscale images and also ...
shparekh's user avatar
  • 820
0 votes
1 answer
353 views

Cryptographically secure random number generator for software entropy in Node.js

I'm trying to generate key pairs (public and private) and I'm looking for cryptographically secure libraries that allow me to generate good entropy (both hardware and software) in a node.js ...
AlexCav's user avatar
  • 29
1 vote
1 answer
361 views

Algorithm to randomize output order of a short array in Go

The key difference between this question and the lots of duplicated answers is that, the input array is short, only 3 elements. -- Say I have an ordered set of int. The size of array is only 3 (or a ...
xpt's user avatar
  • 21.1k
0 votes
0 answers
66 views

How is PNG compressing more than the entropy limit?

I have an image of dimension 2868x4312, whose shannon-entropy is ~6.46. It implies the image can't be compressed beyond ~9.52 MB (2868x4312x6.46/8/1024/1024). But when I save this image as PNG using ...
John Paul's user avatar
  • 101
0 votes
0 answers
78 views

Increase entropy pool by writing to /dev/random (macOS)?

I know there's rng-tools for Linux, but haven't found a port for macOS, so I'm trying to do it manually. I understand you can write to /dev/random to fill the entropy pool (at least in Linux, probably ...
irrbloss's user avatar
  • 257
0 votes
2 answers
25 views

entropy-like measure to quantify the robustness of random forest prediction, using fraction of probability

Imagine I have performed a random forest to classify according to three classes: A, B, and C. My correct get for a given sample is [1, 0, 0], because it belongs to class 1. I have two different models ...
Ilario De Toma's user avatar
0 votes
0 answers
38 views

Cross entropy value higher than entropy

I'm getting a result for entropy that is higher than for cross-entropy, which shouldn't be possible: import numpy as np # Define some sample data y_true = np.array([0.10, 0.40, 0.50]) y_pred = np....
tensai's user avatar
  • 71
0 votes
0 answers
69 views

How to adjust Arithmetic Coding parameters in integer form?

Problem: I'm using the arithmetic coding model based on this PDF paper, which is in integer form to realize an unbounded precision. But the application has some problems: all we calculate is the ...
stackdon'tknow's user avatar
0 votes
0 answers
27 views

How to calculate matrix entropy?

I’m a French student that is taking a look at the matrix entropy ! Could you please explain me how do you manually calculate the entropy of a matrix ? My goal is to calculate it with a python programm....
Arthur Dupent's user avatar
0 votes
2 answers
86 views

Unable to find the cause of segmentation fault of huffman coding to do entropy encoding and decoding in c

I have been looking at the code to do entropy encoding and decoding with of huffman coding in c, and I cannot seem to find the reason why I keep getting segmentation fault. #include <stdio.h> #...
Brian's user avatar
  • 11
0 votes
0 answers
603 views

Spectral entropy of a signal in Python

I wish to find spectral entropy of below sinusoidal signal: ## Impprt the libraries import numpy as np import entropy as ent Fs = 100 ## Sampling frequency (Hz) f = 1 ##...
NN_Developer's user avatar
-1 votes
1 answer
118 views

what is the range of gini impurity when more than 2 classes?

when we are building a decisiontree, we are usually calculating the gini impurity at each node. I am interested to see the range of gini impurity in case of more than 2 classes. Because entropy always ...
Ahmad Dawood's user avatar
0 votes
1 answer
174 views

How to calculate time-frequency domain renyi entropy?

This is the definition of Renyi entropy in the time-frequency domain. I will use this function to calculate the Renyi entropy of a spectrogram. px(t,f) will be the outcome of this spectrogram. For ...
ofk's user avatar
  • 49
0 votes
0 answers
61 views

I don’t get information gain equation in XGBoost

I’m currently studying XGBoost, and I learned that information gain in XGBoost is computed like this: XGBoost information gain What I’m curious is that, previously about information gain, I learned ...
em seoyk's user avatar
0 votes
2 answers
313 views

what R Code to calculate the entropy for each level in a categorical variable

I have quite some categorical variable in my dataset, These variables have more than two levels each. Now i want an R code function (or loop) that can calculate the entropy and information gain for ...
highclef's user avatar
  • 169
0 votes
1 answer
94 views

PDF Entropy calculation

Last time mkl helped me a lot, hopefully he (or someone else) can help me with these questions too. Unfortunately I couldn't get access to the ISO norm (ISO 32000-1 or 32000-2). Are these bytes used ...
Moooz's user avatar
  • 15
0 votes
1 answer
232 views

Shannon entropy function returns -0.0 instead of 0.0

My Shannon entropy function for "aaaaa" should return 0.0 but gives me -0.0. How do I fix that?
sansa's user avatar
  • 21
1 vote
1 answer
54 views

Coverage-adjusted entropy estimation in Python

I need to estimate entropy for many variables in vocabulary data, and some of these have only small samples. I have previously done this using the Chao-Shen entropy estimation in R, but now I would ...
johnbasil's user avatar
0 votes
1 answer
221 views

How to find Entropy or KL divergence between two data?

I have two datasets which are same shape: (576, 450, 5) where 576 is the number of examples, 450 is the time points and 5 is the number of channels. I want to calculate entropy and KL-divergence ...
ali's user avatar
  • 119
0 votes
1 answer
395 views

Total Gini impurity or entropy gain for a scikit-learn decision tree

How can I get the total weighted Gini impurity (or entropy) on a trained decision tree in scikit-learn? For instance, the following code on the titanic dataset, import pandas as pd import matplotlib....
J Prestone's user avatar
0 votes
1 answer
23 views

Burp Sequencer, how to set significance level

I read this article How the randomness tests work and it appears the significance level for the analysis can be configured but I can't find any UI element or documentation to make that change.
asenec4's user avatar
  • 377
3 votes
0 answers
99 views

Cannot identify mistake in computing Shannon entropy

According to this blog, Shannon entropy of the prime numbers approaches log_2(e) ≈ 1.442695 as the limit counting prime numbers approaches infinity. I wanted to see this for myself in Python using ...
just_visiting_xyzabc's user avatar
0 votes
0 answers
158 views

Is there a better method for entropy calculation than shannon entropy for measuring entropy of a PowerShell script?

I thought it would be pretty easy to write a function which measures entropy in a string (or more likely copy / paste a method someone else made) in order to detect encoded PowerShell scripts. Googled ...
J.Todd's user avatar
  • 737
1 vote
0 answers
290 views

How to calculate the KL divergence for two multivariate pandas dataframes

I am training a Gaussian-Process model iteratively. In each iteration, a new sample is added to the training dataset (Pandas DataFrame), and the model is re-trained and evaluated. Each row of the ...
alvarella's user avatar
0 votes
1 answer
23 views

MSEn of data error Mobj unable to be identified

'def MSEn(dataset_mus, Mobj, Scales=3, Methodx='coarse', RadNew=0, Plotx=False): class MSobject: x=(5,2,[3.2]) Mobj = MSobject(x) dataset_mus=np.squeeze(dataset_mus) Func2 = globals()[Methodx.lower()] ...
Mhadm014's user avatar
1 vote
1 answer
212 views

Create relative entropy matrix from pandas dataframe

I have a dataframe of values, say: df = pd.DataFrame(np.array([[0.2, 0.5, 0.3], [0.1, 0.2, 0.5], [0.4, 0.3, 0.3]]), columns=['a', 'b', 'c']) in which every row is a vector of ...
Qubix's user avatar
  • 4,261
1 vote
2 answers
216 views

Contingency matrix to 1D format in Python

2x2 contingency matrix: Cj 2 1 Ci 1 0 Translates to: [[ 0 0 0 1 ] [ 0 0 1 0 ]] The contingency matrix represents the outcome of two clustering algorithms, each with two clusters. The ...
Sean_TBI_Research's user avatar
3 votes
2 answers
222 views

Efficient computation of entropy-like formula (sum(xlogx)) in Python

I'm looking for an efficient way to compute the entropy of vectors, without normalizing them and while ignoring any non-positive value. Since the vectors aren't probability vectors, and shouldn't be ...
Assaf's user avatar
  • 194
0 votes
1 answer
356 views

How many bits are required to encode information in probability set G = [0.001, 0.002, 0.003, 0.994]?

I am currently working on data compression and thought it would be a good time to read up on the basics of information theory to better understand data compression and its algorithms. As I understand, ...
prime_mover's user avatar
0 votes
1 answer
100 views

Adding and removing byte from a file in a loop

I'm a security researcher and am wondering how I could remove the added bytes from a file. This function below adds a byte every 4 bytes. But I'd like to know how I could remove those bytes as well. ...
Travis's user avatar
  • 37
1 vote
1 answer
150 views

Entropy calculation gives NaN - is applying na.omit a valid tweak?

By definition, the entropy is defined as: entropy <- function (p) sum(-p * log(p)) I'm performing LCA using the poLCA package and trying to calculate entropy, which for some of my models are ...
L.Steell's user avatar
1 vote
2 answers
3k views

How to calculate mutual information in PyTorch (differentiable estimator)

I am training a model with pytorch, where I need to calculate the degree of dependence between two tensors (let's say they are the two tensors each containing values very close to zero or one, e.g. v1 ...
Arsalan's user avatar
  • 343
0 votes
0 answers
56 views

Extract possible sample combinations from multiple count constraints

I have some input data like this. unique ID Q1 Q2 Q3 1 1 1 2 2 1 1 2 3 1 0 3 4 2 0 1 5 3 1 2 6 4 1 3 And my target is to extract some data which satisfy the following conditions: total count: ...
cindy50633's user avatar
0 votes
0 answers
37 views

Is the runtime of df.groupby().transform() linear in the number of groups in the groupby object?

BACKGROUND I am calculating racial segregation statistics between and within firms using the Theil Index. The data structure is a multi-indexed pandas dataframe. The calculation involves a lot of df....
JP Ferguson's user avatar

1
2 3 4 5
13