soc logo

"No bird soars too high, if he soars with his own wings."   W. Blake

Rigorous Floating-Point Mixed-Precision Tuning

POPL 2017 screenshot

Abstract

Virtually all real-valued computations are carried out using floating-point data types and operations. The precision of these data types must be set with the goals of reducing the overall round-off error, but also emphasizing performance improvements. Often, a mixed-precision allocation achieves this optimum; unfortunately, there are no techniques available to compute such allocations and conservatively meet a given error target across all program inputs. In this work, we present a rigorous approach to precision allocation based on formal analysis via Symbolic Taylor Expansions, and error analysis based on interval functions. This approach is implemented in an automated tool called FPTuner that generates and solves a quadratically constrained quadratic program to obtain a precision-annotated version of the given expression. FPTuner automatically introduces all the requisite precision up and down casting operations. It also allows users to flexibly control precision allocation using constraints to cap the number of high precision operators as well as group operators to allocate the same precision to facilitate vectorization. We evaluate FPTuner by tuning several benchmarks and measuring the proportion of lower precision operators allocated as we increase the error threshold. We also measure the reduction in energy consumption resulting from executing mixed-precision tuned code on a real hardware platform. We observe significant energy savings in response to mixed-precision tuning, but also observe situations where unexpected compiler behaviors thwart intended optimizations.

Citation

Wei-Fan Chiang, Marek Baranowski, Ian Briggs, Alexey Solovyev, Ganesh Gopalakrishnan, Zvonimir Rakamaric
Rigorous Floating-Point Mixed-Precision Tuning
Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL), 300--315, doi:10.1145/3009837.3009846, 2017.

BibTeX

@inproceedings{2017_popl_cbbsgr,
  title = {Rigorous Floating-Point Mixed-Precision Tuning},
  author = {Wei-Fan Chiang and Marek Baranowski and Ian Briggs and Alexey Solovyev and Ganesh Gopalakrishnan and Zvonimir Rakamaric},
  booktitle = {Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL)},
  publisher = {ACM},
  pages = {300--315},
  doi = {10.1145/3009837.3009846},
  year = {2017}
}

Acknowledgements

We thank Hari Sundar for helping out with the energy measurements, Cindy Rubio-Gonzalez for providing support with Precimonious, Eva Darulova for distributing Rosa and its valuable benchmarks, and the anonymous reviewers for their numerous comments and suggestions. This work was supported in part by NSF awards CCF 1531140, CCF 1643056, and CCF 1552975.