Mathematical Miniatures

Savchev, Svetoslav; Andreescu, Titu

doi:10.5948/upo9780883859575

Cited by 10 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The last line is simply Q(w), showing that QA OCS preserves the original quantization result. To derive the last line, we apply Hermite's Identity (Savchev & Andreescu, 2003) with n = 2:…”

Section: Quantization-aware Splittingmentioning

confidence: 99%

Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

Zhao¹,

Hu²,

Dotzel³

et al. 2019

Preprint

View full text Add to dashboard Cite

Quantization can improve the execution latency and energy efficiency of neural networks on both commodity GPUs and specialized accelerators. The majority of existing literature focuses on training quantized DNNs, while this work examines the less-studied topic of quantizing a floatingpoint model without (re)training. DNN weights and activations follow a bell-shaped distribution post-training, while practical hardware uses a linear quantization grid. This leads to challenges in dealing with outliers in the distribution. Prior work has addressed this by clipping the outliers or using specialized hardware. In this work, we propose outlier channel splitting (OCS), which duplicates channels containing outliers, then halves the channel values. The network remains functionally identical, but affected outliers are moved toward the center of the distribution. OCS requires no additional training and works on commodity hardware. Experimental evaluation on ImageNet classification and language modeling shows that OCS can outperform state-of-the-art clipping techniques with only minor overhead.

show abstract

“…The last line is simply Q(w), showing that QA OCS preserves the original quantization result. To derive the last line, we apply Hermite's Identity (Savchev & Andreescu, 2003) with n = 2:…”

Section: Quantization-aware Splittingmentioning

confidence: 99%

Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

Zhao¹,

Hu²,

Dotzel³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…where the second last equation in Eq. 28 follows [45]. Here we can see that by splitting to K channels, the effect is exactly same as increasing the quantization precision by K times for the original channel, which proves that the channel splitting can reduce the quantization error.…”

Section: B Channel Splitting For Non-bottleneck Layersmentioning

confidence: 59%

Q-LIC: Quantizing Learned Image Compression with Channel Splitting

Sun¹,

Lei²,

Katto³

2022

Preprint

View full text Add to dashboard Cite

Learned image compression (LIC) has reached a comparable coding gain with traditional hand-crafted methods such as VVC intra. However, the large network complexity prohibits the usage of LIC on resource-limited embedded systems. Network quantization is an efficient way to reduce the network burden. This paper presents a quantized LIC (QLIC) by channel splitting. First, we explore that the influence of quantization error to the reconstruction error is different for various channels. Second, we split the channels whose quantization has larger influence to the reconstruction error. After the splitting, the dynamic range of channels is reduced so that the quantization error can be reduced. Finally, we prune several channels to keep the number of overall channels as origin. By using the proposal, in the case of 8-bit quantization for weight and activation of both main and hyper path, we can reduce the BD-rate by 0.61%-4.74% compared with the previous QLIC. Besides, we can reach better coding gain compared with the state-of-the-art network quantization method when quantizing MS-SSIM models. Moreover, our proposal can be combined with other network quantization methods to further improve the coding gain. The moderate coding loss caused by the quantization validates the feasibility of the hardware implementation for QLIC in the future.

show abstract

“…This is a restatement of a theorem of J. Selfridge and E. Straus [30]. A beautiful proof of this theorem is given in [29,Chapter 46,, where it is duly called a masterpiece. For related material, see [30,12,10], and [15, Exercise 29, p. 29].…”

Section: Coincidence Of the Incenter And The Centroid Of An Edge-incementioning

confidence: 99%

Coincidences of Centers of Edge-Incentric, or Balloon, Simplices

Hajja

2006

Result. Math.

View full text Add to dashboard Cite

An edge-incentric d-simplex is defined to be a d-simplex S which admits a (d − 1)-sphere that touches all the edges of S internally. The center of such a sphere is called the edge-incenter of S and is denoted by B. Equivalently, S is edge-incentric if and only if its vertices are the centers of d + 1 (d − 1)-spheres in mutual external touch, and for this reason one may call such an S a balloon d-simplex. An orthocentric d-simplex is a d-simplex in which the altitudes are concurrent. The point of concurrence is called the orthocenter and is denoted by H. The spaces of edge-incentric and of orthocentric d-simplices have the same dimension d in the sense that a d-simplex in either space can be parametrized, up to shape, by d numbers. Edge-incentric and orthocentric tetrahedra are the first two of the four special classes of tetrahedra studied in [1, Chapter IX.B,.The degree of regularity implied by the coincidence of two or more centers of a general d-simplex is investigated in [8], where it is shown that the coincidence of the centroid G, the circumcenter C, and the incenter I does not imply much regularity. For an orthocentric d-simplex S, however, it is proved in [9] that if any two of the centers G, C, I, and H coincide, then S is regular. In this paper, the same question is addressed for edge-incentric d-simplices. Among other things, it is proved that if any three of the centers G, C, I, and B of an edge-incentric d-simplex S coincide, then S is regular, and it is also shown that none of the coincidences G = B, I = B, and I = G implies regularity (except when d ≤ 3, d ≤ 4, and d ≤ 6, respectively). In contrast with the afore-mentioned results for orthocentric d-simplices, this emphasizes once more the feeling that, regarding many important properties, orthocentric d-simplices are the true generalizations of triangles.Several open questions are posed. Mathematics Subject Classification (2000). Primary 52B12; Secondary 52B15, 52B11.

show abstract

Mathematical Miniatures

Cited by 10 publications

References 0 publications

Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

Q-LIC: Quantizing Learned Image Compression with Channel Splitting

Coincidences of Centers of Edge-Incentric, or Balloon, Simplices

Contact Info

Product

Resources

About