Image affine transform

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, LCMark

Locked
monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Image affine transform

Post by monte » Thu Oct 08, 2015 6:29 am

I took a look at the MCImage transforms functions today because Scott Rossi asked about distorting images. As long as he means something we can do with affine transformations it shouldn't be tricky to give him what he wants.

While in there I noticed that these functions apply_transform, rotate_transform, resize_transform and flip_transform appear ripe for refactoring. Specifically I'm not really sure why apply_tranform doesn't just handle all the cases and then anything that changes a property that might impact the transform can just call apply_transform. With a couple of tweaks for efficiency resize_transform covers all the current cases anyway so you could basically drop its code into apply_transform and delete the rest and any calls to rotate_transform could be changed to set angle then apply_transform.
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: Image affine transform

Post by LCMark » Thu Oct 08, 2015 7:45 am

@monte: It could probably be reorganized - whether it would be worth the time invested to ensure all the current fiddly semantics of how image properties affect the transform are unchanged, I'm not sure ;) (all three of those methods just compute an affine transform) We've long been intending to add a 'transform' property which would bypass the calculations called from apply_transform() and allow you to specify it exactly - keeping the current engine semantics untouched, but allowing more flexible transforms as well.

By the way, I think Scott's request is probably almost satisfied by affine transforms - although I suspect he might also be suggesting perspective corrected transforms too.

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: Image affine transform

Post by LCMark » Thu Oct 08, 2015 7:55 am

Ian and I have discussed adding a transform property a number of times - but I can't exactly remember the problems with implementing it. I'll check when he gets in - I think it was related to ensuring things still worked well with the 'rect'.

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: Image affine transform

Post by LCMark » Thu Oct 08, 2015 8:32 am

In terms of actual properties, the general idea would be the following:
  • We add a 'transform' property. This would be a six number list, representing a 'standard' affine transformation including translation (the latter is needed so you can position the image at sub-pixel positions - which is necessary if you want consistent rotation without jiggle).
  • We add properties controlling the specific parts of the transform (essentially an affine transformation can always be decomposed into a rotation, scale, skew and translate) - angle, xScale, yScale, xSkew, ySkew, xTranslate, yTranslate.
  • If a transform is set then the rect is determined from using the transform on the intrinsic size of the image (i.e. its formattedWidth and formattedHeight).
  • If a transform is set and the rect is changed, the transform is changed so that the image fits within the new rect.
There is an issue here with 'angle' being overloaded - this might need a different property name to distinguish between 'setting the angle in the current way' and 'setting the angle in a way where you want to use the transform semantics', but apart from that it is mainly a bit of maths (which I think Ian has already figured out as part of the Canvas module for Widgets). The actual underlying infrastructure is there - the image runs on the m_has_transform and m_transform instance variables.

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: Image affine transform

Post by monte » Thu Oct 08, 2015 9:44 am

Thanks for the explanation. It looks like skia SkMatrix supports perspective.

I'm actually not all that sure we need to worry about exposing a full transform property. I would just change angle to a float and add the skew properties. I think setting the rect and flip covers the rest. Anyone needing more than that is likely playing in the LCB space anyway...

The complicated part with rect is probably working out the new formatted rect after the skew. I'm guessing the logical way to do that is applying the transform to the rect. At the moment for angle it's just calculated the normal way but perhaps that function is reused by the rotate command...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

LCMark
Livecode Staff Member
Livecode Staff Member
Posts: 1206
Joined: Thu Apr 11, 2013 11:27 am

Re: Image affine transform

Post by LCMark » Thu Oct 08, 2015 1:21 pm

@monte: There be dragons lurking here :)

The current behavior of the image object depends on a number of properties:
  • rect
  • formattedWidth / formattedHeight (let's call this the 'intrinsic size').
  • angle
  • lockLocation
If angle and lockLocation are not set, then on open the image resets itself to be the intrinsic size. Subsequent changes to the rect cause scaling.

If angle is not set, and lockLocation is set, then on open the image retains its rect's size (as specified in the rect), and any changes to the width/height change the scale.

If angle is set, then on open the image resets itself to be the bounds of the transformed image at intrinsic size (i.e. the bounds of a rotated rectangle of intrinsic size around the origin). You cannot subsequently set its width / height, just its location.

If angle is set, and lockLocation is set, then a rect of intrinsic size is rotated, and then scaled to fit within the rect. You can subsequently change the width/height, and changing the angle will cause it to still have the same rect, but you'll get skew.

So, basically, at the moment you can actually get any affine transform out of the image object if you know how the maths works - the only thing you cannot do is get a sub-pixel translation which is actually quite important for positioning transformed images and ensuring you don't get 'jitter' when you want to smoothly animate the transformation (and can make them sit side by side without gaps appropriately).

I think it would be quite hard to retain backwards-compatibility (which we have to do) *and* find a way to more easily specify skew (especially as you can, actually already skew images - it would be possible to write a script handler which takes angle, scale, skew and applies the appropriate properties to the image to do it).

The internals of the image object are fully setup to handle a transform property - I think it would be a great deal less work to implement direct (and easy!) access to this and put the image object in a 'new mode' making it crystal clear and easy to transform than try and 'fit in' more properties to manipulate the rather muddy transform calculations which exist.

So - here is how I think it should work:

If transform (or one of its synthetic friends e.g. xScale, yScale etc.) are set then the image object enters the 'new mode'. In the new mode, the rect of the image object is determined by the transform of its 'intrinsic rect' (which is a rect of formattedWidth by formattedHeight sitting with its center on the origin) - this is always the case. To be exact - the rect of an image is always the integer bounding box of the transform(rect(-formattedWidth/2,-formattedHeight/2,formattedWidth/2,formattedHeight/2)).

Changing any of the transform parameters changes the transform and then the rect (obviously).

Now, the more difficult part of this is what to do when the rect is set as this does not contain enough information to have a single solution to the transform - each rect can give rise to a whole family of possible transforms which would result in the requested rect. However, if we make a decision then changing the rect is equivalent to a translation and scale relative to the current transform then that gives a reasonable behavior. Essentially, a transform is calculated which takes things from old-rect to new-rect, concatenated with the current transform, and then sets this as the new transform. I'm pretty confident that the maths would work out okay here - i.e. that this would result in the newly calculated rect being what was asked for.

This probably sounds a lot more complicated than it actually is to implement - it is mainly just a bit of geometry (which I think we already have the code to do).

Indeed, a similar mechanism could be applied to the current graphic object to clean that up - and in so doing fix a number of issues which have plagued it for years in terms of rect's being too big, jitter when setting the angle of certain shapes, the rect of a 1 pixel horizontal line etc.

monte
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 1564
Joined: Fri Jan 13, 2012 1:47 am
Contact:

Re: Image affine transform

Post by monte » Fri Oct 09, 2015 9:13 pm

Yep dragons ;-)

I think I'll look for lower hanging fruit...
LiveCode User Group on Facebook : http://FaceBook.com/groups/LiveCodeUsers/

Locked

Return to “Engine Contributors”