windows - C++ casting to base and "overwriting" vptr issue -

i reading new c++ challenge: http://blogs.msdn.com/b/vcblog/archive/2014/02/04/challenge-vulnerable-code.aspx

the supplied code if full of issues, obvious programming habits, visible c++ natives :-)

it described in comments, 1 particular line (37) particularly dangerous:

imagefactory::debugprintdimensions((imagefactory::customimage*)image); 

the function calls virtual method of customimage (defined first time in customimage).

this allegedly causes first member of customimage treated vptr of instance (it's unique_ptr actually) , make binary pointed treated executable (perhaps malicious) code..

while can understand this, wonder why work.

customimage virtual class, (probably) first 4 bytes (just assume x86) vptr, , unique_ptr member next.. , since cast doesn't seem shifting anything...

... how possible execute data held unique_ptr?

my take (and i'm more happy corrected):

here, customimage polymorphic class (with vptr first "member" under windows abi), imageis not. order of definitions means imagefactory functions know customimage is an image, main() not.

so when factory does:

image* imagefactory::loadfromdisk(const char* imagename) {     return new (std::nothrow) customimage(imagename); } 

the customimage* pointer converted image* , fine. because image not polymorphic, pointer adjusted point (pod) image instance inside customimage -- crutially, after vptr, because comes first in polymorphic class in ms abi (i assume).

however, when to

imagefactory::debugprintdimensions((imagefactory::customimage*)image); 

the compiler sees c-style cast 1 class knows nothing another. take address has , pretend it's customimage* instead. address in fact points image within custom image; because image empty, , presumably empty base class optimisation in effect, ends pointing first member within customimage, i.e. unique_ptr.

now, imagefactory::debugprintdimensions() assumes it's been handed pointer fully-complete customimage, address equal address of vptr. hasn't -- it's been handed address of unique_ptr, because @ point @ called, compiler didn't know better. dereferences thinks vptr (really data we're in control of), looks offset of virtual function , blindly exectutes -- , we're in trouble.

there couple of things have helped mitigate this. firstly, since we're manipulating derived class via base class pointer, image should have virtual destructor. have made image polymorphic, , in probability wouldn't have had problem (and wouldn't leak memory, either).

secondly, because we're casting base-to-derived, dynamic_cast should have been used rather c style cast, have involved run-time check , correct pointer adjustment.

lastly, if compiler had information hand when compiling main(), might have been able warn (or performed cast correctly, adjusting polymorphic nature of customimage). moving class definitions above main() recommended too.