Invertible Coding of Stationary Measures: Connecting Letter and Word Sequences
This paper introduces the "normalized transport," a novel method for bijectively mapping between stationary ergodic measures on sequences of different alphabets (like letters and words) using self-avoiding codes, preserving stationarity and ergodicity.
Generalizing Information Theory: Uncertainty Reduction through Arbitrary Loss Functions
Entropy and information can be generalized beyond Shannon's original framework by considering arbitrary loss functions to quantify uncertainty reduction, rather than just message length. This provides a unified perspective on various information-theoretic quantities.
A Two-Parameter Generalized Tsallis Entropy and Its Information-Theoretic Properties
This article proposes a two-parameter generalization of the Tsallis entropy, called the generalized Tsallis entropy, and derives its fundamental information-theoretic properties, including pseudo-additivity, sub-additivity, joint convexity, and information monotonicity.
Rényi Divergence and Sibson Mutual Information as Measures of α-Leakage in Information-Theoretic Privacy
Rényi divergence and Sibson mutual information are proposed as exact α-leakage measures, quantifying the maximum and average information gain an adversary can obtain on sensitive data through a privacy-preserving channel.
Achievable Rate Analysis of Ordered Reliability Bits Guessing Random Additive Noise Decoding (ORBGRAND) for General Bit Channels and Application in Bit-Interleaved Coded Modulation (BICM)
The paper derives the generalized mutual information (GMI) of ordered reliability bits guessing random additive noise decoding (ORBGRAND) for memoryless binary-input channels with general output conditional probability distributions. The analysis provides insights into understanding the gap between the ORBGRAND achievable rate and the channel mutual information. As an application, the paper studies the ORBGRAND achievable rate for bit-interleaved coded modulation (BICM), showing that the gap is typically small, suggesting the feasibility of ORBGRAND for high-order coded modulation schemes.
Precise Characterization of the Second-Order Randomized Identification Capacity of Additive White Gaussian Noise Channels
The second-order randomized identification capacity of the Additive White Gaussian Noise Channel (AWGNC) has the same form as the second-order transmission capacity, with the only difference being that the maximum number of messages in randomized identification scales double exponentially in the blocklength.
Expurgation Refinement: Achieving the Expurgated Exponent with High Probability
For a wide range of channels and pairwise-independent code ensembles, expurgating an arbitrarily small fraction of codewords from a randomly selected code results in a code that attains the expurgated exponent with high probability.
Analytical Estimation of Average Entropy for Gaussian Mixture Distributions
The authors develop an analytical method to estimate the average differential entropy of a Gaussian mixture distribution, where the component means are i.i.d. Gaussian vectors. They obtain a series expansion in the ratio of the variances of the component means and the shared covariance matrix, providing an approximation with a quantifiable error bound.
A Characterization of Optimal Prefix Codes: Completeness and Strong Monotonicity as Necessary and Sufficient Conditions
A prefix code is optimal if and only if it is complete and strongly monotone.
Characterizing Directed and Undirected Metrics of High-Order Interdependence
The paper investigates the relationship between two practical metrics of high-order interdependencies: the redundancy-synergy index (RSI), which captures directed interdependencies, and the O-information, which captures undirected interdependencies. The results reveal tight links between these two quantities and provide interpretations in terms of likelihood ratios and information geometry.
© 2024 by Linnk AI