Performance/WhatToTest

From Adobe Open Source Wiki
Revision as of 18:38, 29 October 2008 by ChrisCox (Talk | contribs)

Jump to: navigation, search

Contents


What things need to be tested as part of this benchmark?

Fundamental Concepts

Most of these can't easily be tested.

  • regular types
  • basic math
  • basic flow control
  • dereference (pointer/iterator is a regular type that can be dereferenced)
  • type conversion
    • performance of type conversions (known to be a problem on certain compilers)

Language Concepts

  • abstraction
  • inheritance
  • function calls
  • exceptions
  • rtti
  • function objects
  • template instantiation
  • constructors
    • Are empty constructors correctly optimized away?
    • Are copy constructors correctly optimized away?
    • Are default copy constructors efficient?

Simple Idioms

These are building blocks, with more than one way to express them. These are commonly reused things, usually found in headers like "inlines.h", "math_utils.h", etc.

  • swap
  • absolute value
  • min/max
  • switch versus if/else trees
  • pin values to range (max, min combo)
  • rotate bits
  • byte order reversal
  • round up/down
  • round float to int
  • thresholding
  • pointer alignment tests and comparisons

Complex Idioms

  • hand coded memcpy, memmove, memset, memcmp
    • simple types and user defined types
    • arrays and containers
  • safe bool cast from a smart pointer
  • convolution
    • 1D and 2D
  • matrix multiplication
  • min/max of a sequence
  • summation of a sequence
  • sorting a sequence
  • reversing a sequence
  • rotating members of a sequence
  • find/search sequence
  • lookup tables
  • interleave/deinterleave buffers
  • transpose block
  • rotate a block +- 90 and 180 degrees
  • histogram building
  • stack like operations
  • bitarrays and manipulations of them
  • complex type and template

Runtime and Library Support

  • allocation / deletion
    • new, delete
    • malloc, free
  • mathlib
  • string manipulation routines
    • strcpy, strchr, strcat, etc.
  • memory routines
    • memcpy, memmove, memset, memcmp, etc.
  • ctype routines
    • isdigit, isalpha, etc.
  • stdio
    • printf, getc, fread, etc.
  • C++ iostreams
    • cout, ostream, etc.
  • stdlib
    • strtod, strtof,

More Complex Stuff

  • STL containers
  • STL algorithms
  • iostreams

Optimizations

Need to test as many as possible.

  • algebraic simplification
  • constant folding
  • constant propagation
  • copy propagation
  • common subexpression elimination
  • loop invariant code motion
  • dead code elimination
  • strength reduction
  • value range propagation (aka Predicate Simplification?)
  • loop fusion
  • loop interchange
  • loop normalization
  • loop removal
  • loop unrolling
  • loop unswitching
  • normalization of references
  • scalar replacement of arrays
  • scalar replacement of structs
  • instruction scheduling

Conformance to Specification

  • user specified swap in STL algorithms
  • short circuit evaluation of boolean expressions
  • growth policies and complexities of STL containers

Language Extensions

Other ?

  • Binary Size issues
    • Are duplicate templates correctly collapsed?
    • Are unused template member functions correctly removed?
    • Are unused (or duplicate) static, const, and static const values correctly removed?