Faster C++:Move Construction and Perfect Forwarding

Pete IsenseeAdvanced Technology GroupMicrosoft

Problem Statement

• Copying deep objects is expensive• C++ is built on copy semantics– STL containers store by value– Compiler temporaries are copied by value

• Copying is often non-obvious in source code• Games copy objects – a lot!

Example

struct ParticleSystem { std::vector< Particle > mPar; Texture mTex;};

Deeper

ParticleSystem particleSys(...); particleSys = StartExplosion(); // Explosion beginsparticleSys += AddSmoke(); // More particles added

struct Texture { unsigned long mSize; unsigned long* mpBits;};

Deep

Shallow

struct Particle { Vector3 mPos; Vector3 mVel; Color mCol;};

ParticleSystem particleSys(...); particleSys = StartExplosion(); // Explosion beginsparticleSys += AddSmoke(); // More particles added

ParticleSystem particleSys(...);particleSys = StartExplosion(); // Explosion beginsparticleSys += AddSmoke(); // More particles added

particleSys StartExplosion()

tv

t

…

v

t

…

v

t

…

v

tv

t

…

v

tv

…

…

…

t

…

v

Copying Temp Objects is Expensive

1 10 100 1000 10000 100000 10000001

10

100

1000

10000

100000

1000000

Particles

Ticks

ParticlesCopy

(ticks)1 6,113

10 7,201100 5,543

1,000 8,57910,000 56,614

100,000 635,9621,000,000 6,220,013

Perf of operator=(const ParticleSystem&)

Avoiding Temporaries is Difficultbool Connect( const std::string& server, ... );if( Connect( “microsoft.com” ) ) // temporary object created

v.push_back( X(...) ); // another temporary

a = b + c; // b + c is a temporary object

a = b + c + d; // c+d is a temporary object // b+(c+d) is another temporary object

x++; // returns a temporary object

What We Would Like…

• Is a world where…– We could avoid unnecessary copies– In cases where it was safe to do so– Completely under programmer control

• For example…

ParticleSystem particleSys(...);particleSys = StartExplosion(); // Explosion beginsparticleSys += AddSmoke(); // More particles added

particleSys StartExplosion()

tv

tv

t

…

v

tv

tv

t

…

v

tv

…t

…

v

…

Consider Assignmentstruct ParticleSystem { std::vector< Particle > mPar; Texture mTex;};

ParticleSystem& operator=( const ParticleSystem& rhs ) { if( this != &rhs ) { mPar = rhs.mPar; // Vector assignment (copy) mTex = rhs.mTex; // Texture assignment (copy) } return *this;}

Canonical copy assignment

ParticleSystem& operator=( <Magic Type> rhs ) { // move semantics here ... return *this;}

What we want

Solution: C++11 Standard to the Rescue

• Don’t copy when you don’t need to; move instead

• Critically important for deep objects• Key new language feature: rvalue references• Enables move semantics, including– Move construction– Move assignment– Perfect forwarding

Example ParticleSystem& operator=( const ParticleSystem& rhs ) { if( this != &rhs ) { mPar = rhs.mPar; // Particle vector copy mTex = rhs.mTex; // Texture copy } return *this;}

Copy assignment

ParticleSystem& operator=( ParticleSystem&& rhs ) { if( this != &rhs ) { mPar = std::move( rhs.mPar ); // Vector move mTex = std::move( rhs.mTex ); // Texture move } return *this; }

Move assignment

Movable Objects: rvalues• Move from = eviscerate thyself• Every expression is either an lvalue or rvalue• Always safe to move from an rvalue

lvalue rvalueIn memory yes noCan take its address yes noHas a name yes noMoveable no* yes

lvalue/rvalue examples

X x; // x is an lvalue

X(); // X() is an rvalue

int a; // a is an lvalue

int a = 1+2; // a is an lvalue; 1+2 is an rvalue

foo( x ); // x is an lvalue

foo( bar() ); // bar() is an rvalue

++x; // lvalue

“abc” // lvalue

*ptr // lvalue

4321 // rvalue

x+42 // rvalue

x++; // rvalue

std::string( “abc” ) // rvalue

rvalue References

• T&: reference (pre C++11)• T&: lvalue reference in C++11• T&&: rvalue reference; new in C++11• rvalue references indicate objects that can be

safely moved from• rvalue references bind to rvalue expressions• lvalue references bind to lvalue expressions

T&&

Bindingfoo( ParticleSystem&& ); // A: rvaluefoo( const ParticleSystem&& ); // B: const rvaluefoo( ParticleSystem& ); // C: lvaluefoo( const ParticleSystem& ); // D: const lvalue

ParticleSystem particleSys;const ParticleSystem cparticleSys;

foo( particleSys ); // lvaluefoo( StartExplosion() ); // rvaluefoo( cparticleSys ); // const lvalue

Binding and Overload Resolution Rules

ExpressionReference Type

rvalue const rvalue lvalue const lvalue Priority

T&& yes highest

const T&& yes yes

T& yes

const T& yes yes yes yes lowest

std::move

• std::move ~= static_cast< T&& >(t)• Tells compiler: treat this named variable as an rvalue• Highly complex implementation due to reference collapsing,

parameter deduction and other arcane language rules

template< class T > inline typename std::remove_reference<T>::type&&move( T&& t ) noexcept { using ReturnType = typename std::remove_reference<T>::type&&; return static_cast< ReturnType >( t );}

ParticleSystem& operator=( ParticleSystem&& rhs ) { if( this != &rhs ) { mPar = std::move( rhs.mPar ); // Vector move assignment mTex = std::move( rhs.mTex ); // Texture move assignment } return *this; }

Move assignParticleSystem& operator=( ParticleSystem&& rhs ) { if( this != &rhs ) { mPar = std::move( rhs.mPar ); // Vector move assignment mTex = std::move( rhs.mTex ); // Texture move assignment } return *this; }

std::vector<T>& operator=( std::vector<T>&& rhs ) { if( this != &rhs ) { DestroyRange( mpFirst, mpLast ); // call all dtors if( mpFirst != nullptr ) free( mpFirst ); mpFirst = rhs.mpFirst; // eviscerate mpLast = rhs.mpLast; mpEnd = rhs.mpEnd; // rhs now empty shell rhs.mpFirst = rhs.mpLast = rhs.mpEnd = nullptr; } return *this; }

// Standard assignment operatorTexture& Texture::operator=( const Texture& rhs ) { if( this != &rhs ) { if( mpBits != nullptr) free( mpBits ); mSize = rhs.mSize; mpBits = malloc( mSize ); memcpy( mpBits, rhs.mpBits, mSize ); } return *this; }

Texture& Texture::operator=( Texture&& rhs ) { if( this != &rhs ) { if( mpBits != nullptr ) free( mpBits ); mpBits = rhs.mpBits; // eviscerate mSize = rhs.mSize; rhs.mpBits = nullptr; // clear rhs } return *this; }

Intermission• Use rvalue reference semantics to enable

moves• Use non-const rvalue: rhs is reset• std::move tells compiler “this is really an

rvalue”• Binding rules allow gradual conversion– Implement rvalue reference semantics as you go– Start in low-level libraries– Or start in high-level code, your choice

Performance Revisited

ParticlesCopy

(ticks)Move (ticks)

1 6,113 101910 7,201 1100

100 5,543 9681,000 8,579 1200

10,000 56,614 865100,000 635,962 993

1,000,000 6,220,013 1173

1 10 100 1000 10000 100000 10000001

10

100

1000

10000

100000

1000000

Particles

Ticks

operator=(const ParticleSystem&)

operator=(ParticleSystem&&)

Move ConstructorsParticleSystem::ParticleSystem( ParticleSystem&& rhs ) : // invoke member move ctors mPar( std::move( rhs.mPar ) ), mTex( std::move( rhs.mTex ) ) {} vector<T>::vector( vector<T>&& rhs ) :

mpFirst( rhs.mpFirst ), // eviscerate mpLast ( rhs.mpLast ), mpEnd ( rhs.mpEnd ){ // rhs now an empty shell rhs.mpFirst = rhs.mpLast = rhs.mpEnd = nullptr; }

Texture::Texture( Texture&& rhs ) : mpBits( rhs.mpBits ), // eviscerate mSize( rhs.mSize ){ // rhs now an empty shell rhs.mpBits = nullptr; }

Perfect Forwarding ProblemSuppose we have some setter functions

void ParticleSystem::SetTexture( const Texture& texture ) { mTex = texture; // We’d like to move if tx is a temporary}

void ParticleSystem::SetTexture( Texture&& texture ) { mTex = std::move( texture ); // Move}

void ParticleSystem::Set( const A& a, const B& b ) { // Uh-oh, we need three new overloads...}

Func Templates Plus rvalues to the RescuePowerful new rule in C++11. Given:

Template rvalue ref param binds to anythingtemplate< typename T > void f( T&& t ); // template function

ExpressionReference Type rvalue const rvalue lvalue const lvalue Priority

template T&& yes yes yes yes highestT&& yesconst T&& yes yesT& yesconst T& yes yes yes yes lowest

Binding rvalue Reference Template Params

Examplestemplate< typename T > void f( T&& t ); // template function

int a;const int ca = 42;

f( a ); // instantiates f( int& );f( ca ); // instantiates f( const int& );f( StartExplosion() ); // instantiates f( ParticleSystem&& );

Perfect Forwardingtemplate< typename T >void ParticleSystem::SetTexture( T&& texture ) { mTex = std::forward<T>( texture ); // invokes right overload}

template< class T > inline T&& // typical std::forward implementationforward( typename identity<T>::type& t ) noexcept { return static_cast<T&&>( t );}

std::forward<T> equivalent to– static_cast<[const] T&&>(t) when t is an rvalue– static_cast<[const] T&>(t) when t is an lvalue

Perfect Constructors

ParticleSystem::ParticleSystem( const std::vector<Particle>& par, const Texture& texture ) : mPar( par ), mTex( texture ) {}

Typical multi-arg ctor; doesn’t handle rvalues

template< typename V, typename T >ParticleSystem::ParticleSystem( V&& par, T&& texture ) : mPar( std::forward<V>( par ) ), mTex( std::forward<T>( texture ) ) {}

Perfect constructor; handles everything you throw at it!

Implicit Special Member Functions

• Rule of Three: if you define any of the first three, define all• Rule of Two Moves: If you define either move, define both

Function Implicitly generated whenDefault ctor no other ctor explicitly declaredCopy ctor no move ctor or move assign explicitly declaredCopy assign no move ctor or move assign explicitly declaredMove ctor no copy ctor, move assign or dtor explicitly

declaredMove assign no copy ctor, copy assign or dtor explicitly declaredDtor no dtor explicitly declared

Be Explicit About Implicit Special Functionsstruct ParticleSystem { std::vector< Particle > mPar; // Copyable/movable object Texture mTex; // Copyable/movable object

// Ctors ParticleSystem() = delete; ParticleSystem( const ParticleSystem& ) = default; ParticleSystem( ParticleSystem&& ) = default; // Assign ParticleSystem& operator=( const ParticleSystem& ) = default; ParticleSystem& operator=( ParticleSystem&& ) = default; // Destruction ~ParticleSystem() = default; };

C++11• STL containers move enabled– Including std::string

• STL algorithms move enabled– Including sort, partition, swap

• You get immediate speed advantages simply by recompiling

template< typename T >swap( T& a, T& b ) { T tmp( std::move( a ) ); a = std::move( b ); b = std::move( tmp );}

Recommended Idioms: Moveable Typesstruct Deep { Deep( const Deep& ); // Copy ctor Deep( Deep&& ); // Move ctor template< typename A, typename B > Deep( A&&, B&& ); // Perfect forwarding ctor

Deep& operator=( const Deep& ); // Copy assignment Deep& operator=( Deep&& ); // Move assignment ~Deep();

template< typename A > // Deep setters void SetA( A&& );};

Recommended Idioms: Raw PointersT( T&& rhs ) : ptr( rhs.ptr ) // eviscerate{ rhs.ptr = nullptr; // rhs: safe state}

T& operator=( T&& rhs ) { if( this != &rhs ) { if( ptr != nullptr ) free( ptr ); ptr = rhs.ptr; // eviscerate rhs.ptr = nullptr; // rhs: safe state } return *this;}

Move ctor

Move assignment

Recommended Idioms: Higher Level ObjsT( T&& rhs ) : base( std::move( rhs ) ), // base m ( std::move( rhs.m ) ) // members{}

T& operator=( T&& rhs ) { if( this != &rhs ) { m = std::move( rhs.m ); // eviscerate } return *this;}

Move ctor

Move assignment

Recommended Idioms: Perfect Forwardingtemplate< typename A, typename B >T( A&& a, B&& b ) : // binds to any 2 params ma( std::forward<A>( a ) ), mb( std::forward<B>( b ) ){}

template< typename A >void SetA( A&& a ) // binds to anything{ ma = std::forward<A>( a );}

Ctor

Setter

Compilers and Move SupportFeature Microsoft GCC Intel Clangrvalue references VS 2010 4.3 11.1 2.9STL move semantics VS 2010 4.3 11.1 2.9nullptr VS 2010 4.6 12.1 2.9variadic templates 4.3 12.1 2.9defaulted/deleted funcs 4.4 12.0 3.0noexcept 4.6 3.0

Exhaustive list: http://wiki.apache.org/stdcxx/C++0xCompilerSupport

High Level Takeaways

• By overloading on rvalue references, you can branch at compile time on the condition that x is moveable (a temporary object) or not

• You can implement the overloading gradually• Benefits accrue to deep objects• Performance improvements can be significant

Further Research: Topics I Didn’t Cover• xvalues, glvalues, prvalues• Emplacement (e.g. “placement insertion”)– Create element within container, w/ no moves/copies– Uses perfect forwarding and variadic functions

• Other scenarios where moving lvalues is OK• Moves and exceptions• Perfect forwarding not always so perfect– e.g. integral and pointer types; bitfields, too

• noexcept and implicit move

Best Practices• Update to compilers that support rvalue references• Return by value is now reasonable – both readable and fast• Add move ctor/assignment/setters to deep objects• Move idiom: this = rhs pointers, rhs pointers = null• Use non-const rvalue references• When moving, satisfy moved-from obj invariants• Avoid return by const T – prevents move semantics• Be explicit about implicit special functions• Step thru new move code to ensure correctness

Thanks!• Contact me: pkisensee@msn.com • Slides: http://www.tantalon.com/pete.htm • Scott Meyers: http://www.aristeia.com • Stephan Lavavej: http://blogs.msdn.com • Dave Abrahams: http://cpp-next.com • Thomas Becker: http://thbecker.net• Marc Gregoire: http://www.nuonsoft.com

• Let me know what kind of results you see when you move enable your code

Additional Reference Material

C++ Standard References

• N1610 (v0.1) 2004 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2004/n1610.html

• N2118 (v1.0) 2006 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2118.html

• N2844 (v2.0) 2009 (VC10 impl) http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2844.html

• N3310 (sections 840, 847, 858) (v2.1) 2011 (VC11 impl) http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html

• N3053 (v3.0) 2010 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3053.html

rvalue References• Scott Meyers’ Move Semantics and rvalue References:

http://www.aristeia.com/TalkNotes/ACCU2011_MoveSemantics.pdf and http://skillsmatter.com/podcast/home/move-semanticsperfect-forwarding-and-rvalue-references

• Scott Meyers’ Adventures in Perfect Forwarding (C++ and Beyond 2011)• Thomas Becker’s rvalue References Explained:

http://thbecker.net/articles/rvalue_references/section_01.html • STL’s blog:

http://blogs.msdn.com/b/vcblog/archive/2009/02/03/rvalue-references-c-0x-features-in-vc10-part-2.aspx?PageIndex=3

• Marc Gregoire’s Blog http://www.nuonsoft.com/blog/2009/06/07/the-move-constructor-in-visual-c-2010/

• C++11 Features in Visual Studio C++ 11 http://blogs.msdn.com/b/vcblog/archive/2011/09/12/10209291.aspx

• Mikael Kilpelainen’s Lvalues and Rvalues http://accu.org/index.php/journals/227• Moving from lvalues http://cpp-next.com/archive/2009/09/move-it-with-rvalue-references • Binary operators http://cpp-next.com/archive/2009/09/making-your-next-move/ • Emplacement http://stackoverflow.com/questions/4303513/push-back-vs-emplace-back