Difference between revisions of "Performance/WhatToTest"

From Adobe Open Source Wiki
Jump to: navigation, search
(Complex Idioms)
(Runtime and Library Support)
Line 101: Line 101:
  
 
* mathlib
 
* mathlib
 +
** all functions
 +
** compared to inline versions of some functions
  
 
* string manipulation routines
 
* string manipulation routines
** strcpy, strchr, strcat, etc.
+
** strcpy, strcmp, strchr, strcat, etc.
 +
** strncpy, strncmp, strncat, etc.
  
 
* memory routines
 
* memory routines
Line 112: Line 115:
  
 
* stdio
 
* stdio
** printf, getc, fread, etc.
+
** printf, getc, fread, fwrite, etc.
  
 
* C++ iostreams
 
* C++ iostreams
 
** cout, ostream, etc.
 
** cout, ostream, etc.
 +
** compare cout versus printf
  
 
* stdlib
 
* stdlib
** strtod, strtof,
+
** strtod, strtof
  
 
== More Complex Stuff ==
 
== More Complex Stuff ==

Revision as of 05:14, 12 November 2008

Contents


What things need to be tested as part of this benchmark?

If you have additions or suggestions, please add them.

Fundamental Concepts

Most of these can't easily be tested.

  • regular types
  • basic math
  • basic flow control
  • dereference (pointer/iterator is a regular type that can be dereferenced)
  • type conversion
    • performance of type conversions (known to be a problem on certain compilers)

Language Concepts

  • abstraction - what happens to performance when I put { } around a value?
  • inheritance - what happens if my function is in a child class, virtual, or both?
  • function calls - how expensive are different styles of function call?
    • inline, function pointer, defined in another module, etc.
  • exceptions - what is the cost of using exceptions?
    • throwing versus not
    • compared to error return codes
    • exceptions enabled or not in the compiler
  • RTTI - what are the costs?
    • compared to per-object ids or strings
  • function objects - what are the costs/benefits compared to inline functions and function pointers?
  • template instantiation
    • problems with duplicate template bodies and code size?
    • problems with recursive templates or deep usage of templates?
  • constructors
    • Are empty constructors correctly optimized away?
    • Are copy constructors correctly optimized away?
    • Are default copy constructors efficient?

Simple Idioms

These are building blocks, with more than one way to express them. These are commonly reused things, usually found in headers like "inlines.h", "math_utils.h", etc.

For all of these: What is the optimal implementation, and does the compiler recognize other implementations and substitute the optimal one?


  • swap
    • Does the template library correctly handle user defined swap functions?
  • absolute value
  • min/max
  • switch versus if/else trees
  • pin values to range (max, min combo)
  • rotate bits
  • byte order reversal
  • round up/down
  • round float to int
  • thresholding
  • pointer alignment tests and comparisons
  • fixed point math
  • extended precision math (ie: 64 bit on a 32 bit CPU)

Complex Idioms

  • safe bool cast from a smart pointer
  • hand coded memcpy, memmove, memset, memcmp
    • simple types and user defined types
    • arrays and containers
  • convolution
    • 1D, 2D, and more?
  • matrix multiplication
    • 2D is common
    • higher dimensions get a bit esoteric
  • min/max of a sequence
  • summation of a sequence
  • product of a sequence
  • dot product of sequences
  • sorting a sequence
  • reversing a sequence
  • rotating members of a sequence
  • find/search sequence
  • lookup tables
  • interleave/deinterleave buffers
  • transpose block
  • rotate a 2D block
    • plus or minus 90 degrees
    • 180 degrees
    • horizontal flip
    • vertical flip
    • transpose
  • histogram building
  • stack like operations
  • bitarrays and manipulations of them
  • complex type and template

Runtime and Library Support

  • allocation / deletion
    • new, delete
    • malloc, free
  • mathlib
    • all functions
    • compared to inline versions of some functions
  • string manipulation routines
    • strcpy, strcmp, strchr, strcat, etc.
    • strncpy, strncmp, strncat, etc.
  • memory routines
    • memcpy, memmove, memset, memcmp, etc.
  • ctype routines
    • isdigit, isalpha, etc.
  • stdio
    • printf, getc, fread, fwrite, etc.
  • C++ iostreams
    • cout, ostream, etc.
    • compare cout versus printf
  • stdlib
    • strtod, strtof

More Complex Stuff

  • STL containers
  • STL algorithms
  • iostreams

Optimizations

Need to test as many as possible.

  • algebraic simplification
  • constant folding
  • constant propagation
  • copy propagation
  • common subexpression elimination
  • loop invariant code motion
  • dead code elimination
  • strength reduction
  • value range propagation (aka Predicate Simplification?)
  • loop fusion
  • loop interchange
  • loop normalization
  • loop removal
  • loop unrolling
  • loop unswitching
  • normalization of references
  • scalar replacement of arrays
  • scalar replacement of structs
  • instruction scheduling

Conformance to Specification

  • user specified swap in STL algorithms
  • short circuit evaluation of boolean expressions
  • growth policies and complexities of STL containers

Binary Size

  • Are duplicate templates correctly collapsed?
  • Are unused template member functions correctly removed?
  • Are unused (or duplicate) static, const, and static const values correctly removed?

Language Extensions

Other ?