AS2 Pixel HitTest

Whether you knew it or not, hitTests in Flash have a dark secret. They do not actually check to see if the two objects are touching. Instead, it draws a rectangle that fits perfectly around one object, and then another rectangle that fits perfectly around the other object. Then, it tests if THOSE are touching. This can cause some really ugly collision detection goofs.

This is a BAD hitTest:

Wow. You can see just how horrible that was. But luckily, there are ways to fix this!

A good hitTest: (what we'll make)

That was pretty good, eh? Now, let's find out how to do this!

The code

Firstly, in the actions for the root timeline, paste in the following:

import flash.geom.Rectangle;
import flash.display.BitmapData;
import flash.geom.ColorTransform;
import flash.geom.Matrix;
import flash.geom.Point;
_global.checkForCollision = function(p_clip1:MovieClip, p_clip2:MovieClip, p_alphaTolerance:Number):Rectangle  {
	if (_root.stageMaskRect) {
		//make it fill the screen.
		_root.stageMaskRect._width = Stage.width;
		_root.stageMaskRect._height = Stage.height;
		_root.stageMaskRect._x = 0;
		_root.stageMaskRect._y = 0;
	} else {
		//We need to CREATE that movieclip!
		_root.createEmptyMovieClip("stageMaskRect", 0);
		_root.stageMaskRect.beginFill(0xFF0000, 0);
		_root.stageMaskRect.moveTo(0, 0);
		_root.stageMaskRect.lineTo(Stage.width, 0);
		_root.stageMaskRect.lineTo(Stage.width, Stage.height);
		_root.stageMaskRect.lineTo(0, Stage.height);
		_root.stageMaskRect.endFill();
		_root.stageMaskRect._x = 0;
		_root.stageMaskRect._y = 0;
	}
	// set up default params:
	if (p_alphaTolerance == undefined) {
		p_alphaTolerance = 255;
	}
	// get bounds:   
	var bounds1:Object = p_clip1.getBounds(_root);
	var bounds2:Object = p_clip2.getBounds(_root);
	// rule out anything that we know can't collide:
	if (((bounds1.xMax<bounds2.xMin) || (bounds2.xMax<bounds1.xMin)) || ((bounds1.yMax<bounds2.yMin) || (bounds2.yMax<bounds1.yMin))) {
		return null;
	}
	// determine test area boundaries:   
	var bounds:Object = {};
	bounds.xMin = Math.max(bounds1.xMin, bounds2.xMin);
	bounds.xMax = Math.min(bounds1.xMax, bounds2.xMax);
	bounds.yMin = Math.max(bounds1.yMin, bounds2.yMin);
	bounds.yMax = Math.min(bounds1.yMax, bounds2.yMax);
	// set up the image to use:
	var img:BitmapData = new BitmapData(_root.stageMaskRect._width, _root.stageMaskRect._height, false);
	// draw in the first image:
	var mat:Matrix = p_clip1.transform.concatenatedMatrix;
	mat.tx -= bounds.xMin;
	mat.ty -= bounds.yMin;
	img.draw(p_clip1, mat, new ColorTransform(1, 1, 1, 1, 255, -255, -255, p_alphaTolerance));
	// overlay the second image:
	mat = p_clip2.transform.concatenatedMatrix;
	mat.tx -= bounds.xMin;
	mat.ty -= bounds.yMin;
	img.draw(p_clip2, mat, new ColorTransform(1, 1, 1, 1, 255, 255, 255, p_alphaTolerance), "difference");
	// find the intersection:
	var intersection:Rectangle = img.getColorBoundsRect(0xFFFFFFFF, 0xFF00FFFF);
	// if there is no intersection, return null:
	if (intersection.width == 0) {
		return null;
	}
	// adjust the intersection to account for the bounds:   
	intersection.x += bounds.xMin;
	intersection.y += bounds.yMin;
	return intersection;
};
MovieClip.prototype.hitTestShape = function(mc) {
	// check for collisions:
	var collisionRect:Rectangle = checkForCollision(this, mc, 120);
	//Did it return a value?
	if (collisionRect) {
		return true;
	} else {
		return false;
	}
};

That code allows us to use the function hitTestShape rather than the traditional hitTest. Now, all you have to do to check for a real hitTest is this:

if (movieclip1.hitTestShape(movieclip2)) {
	//do stuff.
}

How it Works

Pretty much, what this does is it uses the bitmap version of hitTest. In actionscript 2, you were probably aware of the two movieclip hittests (for an object and for a point.) But you can do another type of hittest between bitmap objects too. With this, you can get a really accurate hitTest because it can easily tell if any of the pixels are overlapping. This is the method that other game development platforms tend to use.

Anyway, this code basically creates bitmap objects with the coordinates of the two movieclips and tests if they overlap.

Credits

The original code for this was made by Grant Skinner. Here's his original copyright statement:

/**
* GTween by Grant Skinner. Aug 1, 2005
* Visit www.gskinner.com/blog for documentation, updates and more free code.
*
*
* Copyright (c) 2005 Grant Skinner
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
**/

Now, what I did is I took his original code and made a few modifications. Firstly, his code returns a shape rather than a boolean value, meaning you couldn't just write if (bob.hitTestShape(jeff)) { and expect it to work. I fixed that, while still allowing you to use the old functionality by creating a second function to do that too.

I also fixed a major glitch: Before, he made it so that all the coordinates were based on the size of the stage that you set when you first made your Flash movie. But if you use something like a vcam (or if you resize the flash player window during runtime) the actual stage size changes, but the Stage.width and Stage.height properties don't change. This really messes up his code. So instead, I made it automatically create an invisible rectangle the same size as the stage. This way, when the stage is scaled, the script can accurately read the true coordinates.

That's all, folks!

Hopefully you can use this in your next game or something. I haven't actually made any new games since I finished debugging Grant's old script, but it seems like the sort of thing that can be really useful. I'm definitely going to make use of this in the next version of Sideview Engine. :D

</david>