This page is dedicated to explaining the logic behind
pychubby relies on the standard 68 facial landmarks framework. Specifically,
dlib model is used to achieve this task. See
pychubby.data for credits
and references. Once the landmarks are detected
one can query them via their index. Alternatively, for the ease of defining new actions
pychubby.detect.LANDMARK_NAMES defines a name for each of the 68 landmarks.
pychubby.detect.LandmarkFace is one of the most important classes that
To construct a
LandmarkFace one needs to provide
- Image of the face
- 68 landmark points
Rather than using the lower level constructor the user will mostly create instances through the class
estimate which detect the landmark points automatically.
Once instantiated, one can use actions (
pychubby.actions.Action) to generate a new (warped)
pychubby.detect.LandmarkFaces is a container holding multiple instances of
additionally provides functionality that allows for performing the
Multiple action on them.
Action is a specific warping recipe that might depend on some parameters. Once instantiated,
one can use their
perform method to warp a
LandmarkFace. To see already available
actions go to Gallery or read how to create your own actions Custom Actions.
In general, faces in images appear in different positions, angles and sizes. Defining actions purely based on coordinates of a given face in a given image is not a great idea. Mainly for two reasons:
- Resizing, cropping, rotating, etc of the image will render the action useless
- These actions are image specific and cannot be applied to any other image. One would be better off using some graphical interface.
One way to solve the above issues is to first transform all the landmarks into some reference space,
define actions in this reference space and then map it back into the original domain.
defines these reference spaces in
pychubby.reference module. Each reference space needs to implement
the following three methods:
The default reference space is the
DefaultRS and its logic is captured in the below figure.
Five selected landmarks are used to estimate an affine transformation between the reference and input space. This trasformation is endcoded in a 2 x 3 matrix A. Transforming from reference to input space and vice versa is then just a simple matrix multiplication.
Displacement field represents a 2D to 2D transformations between two images.
To instantiate a
pychubby.base.DisplacementField one can either use the standard
Alternatively, one can use a factory method
generate that creates a
DisplacemetField based on
displacement of landmark points.