Swift - добавить карнавальную маску к фотографии с лицом.
У меня есть фото с лицом.
У меня есть карнавальная маска:
С помощью этой функции я обнаруживаю лицо:
let ciImage = CIImage(cgImage: photo)
let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
let faces = faceDetector.features(in: ciImage)
if let face = faces.first as? CIFaceFeature {
}
Как обнаружить дыры в маске?
Как мне надеть маску на лицо после обнаружения дырок в маске?
1 ответ
Решение
Я бы, наверное, попробовал такой подход:
Получите leftEyePosition, rightEyePosition и значение faceAngle. (Вся часть CIFaceFeature)
Рассчитайте расстояние между левым и правым глазом.
Вот ссылка на то, как рассчитать расстояние: https://www.hackingwithswift.com/example-code/core-graphics/how-to-calculate-the-distance-between-two-cgpoints
Создайте константы с исходными размерами маски, а также расстоянием по осям x и y до центра одного из глаз.
С расстоянием между глазами вы пропорционально рассчитываете новую ширину вашей маски.
Так вы получите маску нужного размера. Таким же образом вычислите новые расстояния x и y до центра одного из глаз маски.
Снова отрегулируйте все значения пропорционально, чтобы они соответствовали окончательному заданному размеру на экране.
Поместите маску на фотографию, используя координаты глаз, смещая маску глаз до углового расстояния.
Используйте значение faceAngle, чтобы повернуть маску.
Перед тем как импортировать маску в проект, сконвертируйте ее в png с прозрачным фоном, удалите белый фон. Вы можете сделать это в коде, но это потребует много работы, и в зависимости от исходного файла масок это может не сработать.
ОБНОВЛЕНИЕ, я пробовал свое решение. Это простое одноэкранное приложение для iOS, просто скопируйте код в файл ViewController.swift, добавьте свою маску как png и фотографию лица как photo.jpg в проект, и все должно работать.
Вот ссылка на ваше фото в формате png, если вы хотите попробовать:
import UIKit
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
imageMethod()
}
func imageMethod() {
let uiMaskImage = UIImage(named: "QPTF1.png") //converted to png with transperancy before adding to the project
let maskOriginalWidth = CGFloat(exactly: 655.0)!
let maskOriginalHeight = CGFloat(exactly: 364.0)!
let maskOriginalEyeDistance = CGFloat(exactly: 230.0)! //increase or decrease value to change the final size of the mask
let maskOriginalLeftEyePossitionX = CGFloat(exactly: 203.0)! //increase or decrease to fine tune mask possition on x axis
let maskOriginalLeftEyePossitionY = CGFloat(exactly: 200.0)! //increase or decrease to fine tune mask possition on y axis
//This code assumes the image AND face orientation is always matching the same orientation!
//The code needs to be adjusted for other cases using UIImage.Orientation to get the orientation and adjusts the coordinates accordingly.
//CIDetector might also not detect faces which don't have the same orientation as the photo. Try to use CIDetectorImageOrientation to look for other orientations of no face has been detected.
//Also you might want to use other orientation points and scale values (right eye, nose etc.) in case the left eye, and left to right eye distance is not available.
//Also this code is very wordy, pretty sure it can be cut down to half the size and made simpler on many places.
let uiImageFace = UIImage(named: "photo.jpg")
let ciImageFace = CIImage(image: uiImageFace!)
let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
let faces = faceDetector.features(in: ciImageFace!)
if let face = faces.first as? CIFaceFeature {
/*
Getting the distances and angle based on the original photo
*/
let faceAngle = face.faceAngle
let rotationAngle = CGFloat(faceAngle * .pi / 180.0)
//The distance in between the eyes of the original photo.
let originalFaceEyeDistance = CGPointDistance(from: face.leftEyePosition, to: face.rightEyePosition)
/*
Adjusting the mask and its eye coordinates to fit the original photo.
*/
//Setting the scale mask : original.
let eyeDistanceScale = maskOriginalEyeDistance / originalFaceEyeDistance
//The new dimensions of the mask.
let newMaskWidth = maskOriginalWidth/eyeDistanceScale
let newMaskHeight = maskOriginalHeight/eyeDistanceScale
//The new mask coordinates of the left eye in relation to the original photo.
let newMaskLeftEyePossitionX = maskOriginalLeftEyePossitionX / eyeDistanceScale
let newMaskLeftEyePossitionY = maskOriginalLeftEyePossitionY / eyeDistanceScale
/*
Adjusting the size values to fit the desired final size on the screen.
*/
//Using the width of the screen to calculate the new scale.
let screenScale = uiImageFace!.size.width / view.frame.width
//The new final dimensions of the mask
let scaledToScreenMaskWidth = newMaskWidth / screenScale
let scaledToScreenMaskHeight = newMaskHeight / screenScale
//The new final dimensions of the photo.
let scaledToScreenPhotoHeight = uiImageFace!.size.height / screenScale
let scaledToScreenPhotoWidth = uiImageFace!.size.width / screenScale
//The new eye coordinates of the photo.
let scaledToScreenLeftEyeFacePositionX = face.leftEyePosition.x / screenScale
let scaledToScreenLeftEyeFacePositionY = (uiImageFace!.size.height - face.leftEyePosition.y) / screenScale //reversing the y direction
//The new eye to corner distance of the mask
let scaledToScreenMaskLeftEyeX = newMaskLeftEyePossitionX / screenScale
let scaledToScreenMaskLeftEyeY = newMaskLeftEyePossitionY / screenScale
//The final coordinates for the mask
let adjustedMaskLeftEyeX = scaledToScreenLeftEyeFacePositionX - scaledToScreenMaskLeftEyeX
let adjustedMaskLeftEyeY = scaledToScreenLeftEyeFacePositionY - scaledToScreenMaskLeftEyeY
/*
Showing the image on the screen.
*/
let baseImageView = UIImageView(image: uiImageFace!)
//If x and y is not 0, the mask x and y need to be adjusted too.
baseImageView.frame = CGRect(x: CGFloat(exactly: 0.0)!, y: CGFloat(exactly: 0.0)!, width: scaledToScreenPhotoWidth, height: scaledToScreenPhotoHeight)
view.addSubview(baseImageView)
let maskImageView = UIImageView(image: uiMaskImage!)
maskImageView.frame = CGRect(x: adjustedMaskLeftEyeX, y: adjustedMaskLeftEyeY, width: scaledToScreenMaskWidth, height: scaledToScreenMaskHeight)
maskImageView.transform = CGAffineTransform(rotationAngle: rotationAngle)
view.addSubview(maskImageView)
}
}
func CGPointDistanceSquared(from: CGPoint, to: CGPoint) -> CGFloat {
return (from.x - to.x) * (from.x - to.x) + (from.y - to.y) * (from.y - to.y)
}
func CGPointDistance(from: CGPoint, to: CGPoint) -> CGFloat {
return sqrt(CGPointDistanceSquared(from: from, to: to))
}
}
Результат:
Вот мой раскомментированный подход к сканированию глаз. В нем все еще есть некоторые причуды, но они должны быть отправной точкой.
import UIKit
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
imageMethod()
}
func imageMethod() {
struct coords {
let coord: (x: Int, y: Int)
let size: Int
}
let uiMaskImage = UIImage(named: "QPTF1.png") //converted to png with transperancy before adding to the project
let uiMaskImage2 = UIImage(named: "QPTF1.png")
let ciMaskImage2 = CIImage(image: uiMaskImage2!)
let context = CIContext(options: nil)
let cgMaskImage = context.createCGImage(ciMaskImage2!, from: ciMaskImage2!.extent)
let pixelData = cgMaskImage!.dataProvider!.data
let data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
let alphaLevel: CGFloat = 0.0 //0.0 - 1.0 set higher to allow images with partially transparent eyes, like sunglasses.
var possibleEyes: [coords] = []
let frame = 10
var detailLevel = 6
let sizeX = Int((uiMaskImage?.size.width)!)
let sizeY = Int((uiMaskImage?.size.height)!)
var points: [(x: Int, y: Int)] = []
var pointA_X = sizeX / 4
var pointA_Y = sizeY / 4
var pointB_X = sizeX / 4
var pointB_Y = sizeY * 3 / 4
var pointC_X = sizeX * 3 / 4
var pointC_Y = sizeY / 4
var pointD_X = sizeX * 3 / 4
var pointD_Y = sizeY * 3 / 4
var nextXsmaller = pointA_X / 2
var nextYsmaller = pointA_Y / 2
points.append((x: pointA_X, y: pointA_Y))
points.append((x: pointB_X, y: pointB_Y))
points.append((x: pointC_X, y: pointC_Y))
points.append((x: pointD_X, y: pointD_Y))
func transparentArea(_ x: Int, _ y: Int) -> Bool {
let pos = CGPoint(x: x, y: y)
let pixelInfo: Int = ((Int(uiMaskImage2!.size.width) * Int(pos.y)) + Int(pos.x)) * 4
let a = CGFloat(data[pixelInfo+3]) / CGFloat(255.0)
if a <= alphaLevel {
return true
} else {
return false
}
}
func createPoints(point: (x: Int, y: Int)) {
pointA_X = point.x - nextXsmaller
pointA_Y = point.y - nextYsmaller
pointB_X = point.x - nextXsmaller
pointB_Y = point.y + nextYsmaller
pointC_X = point.x + nextXsmaller
pointC_Y = point.y - nextYsmaller
pointD_X = point.x + nextXsmaller
pointD_Y = point.y + nextYsmaller
points.append((x: pointA_X, y: pointA_Y))
points.append((x: pointB_X, y: pointB_Y))
points.append((x: pointC_X, y: pointC_Y))
points.append((x: pointD_X, y: pointD_Y))
}
func checkSides(point: (x: Int, y: Int)) {
var xNeg = (val: 0, end: false)
var xPos = (val: 0, end: false)
var yNeg = (val: 0, end: false)
var yPos = (val: 0, end: false)
if transparentArea(point.x, point.y) {
xNeg.val = point.x
xPos.val = point.x
yNeg.val = point.y
yPos.val = point.y
while true {
if transparentArea(xNeg.val, point.y) {
xNeg.val -= 1
if xNeg.val <= frame {
break
}
} else {
xNeg.end = true
}
if transparentArea(xPos.val, point.y) {
xPos.val += 1
if xPos.val >= sizeX-frame {
break
}
} else {
xPos.end = true
}
if transparentArea(point.x, yNeg.val) {
yNeg.val -= 1
if yNeg.val <= frame {
break
}
} else {
yNeg.end = true
}
if transparentArea(point.x, yPos.val) {
yPos.val += 1
if yPos.val >= sizeY-frame {
break
}
} else {
yPos.end = true
}
if xNeg.end && xPos.end && yNeg.end && yPos.end {
let newEyes = coords(coord: (point.x, point.y), size: (xPos.val - xNeg.val) * (yPos.val - yNeg.val) )
possibleEyes.append(newEyes)
break
}
}
}
}
while detailLevel > 0 {
print("Run: \(detailLevel)")
for (index, point) in points.enumerated().reversed() {
//checking if the point is inside of an transparent area
checkSides(point: point)
points.remove(at: index)
if detailLevel > 1 {
createPoints(point: point)
}
}
detailLevel -= 1
nextXsmaller = nextXsmaller / 2
nextYsmaller = nextYsmaller / 2
}
possibleEyes.sort { $0.coord.x > $1.coord.x }
var rightEyes = possibleEyes[0...possibleEyes.count/2]
var leftEyes = possibleEyes[possibleEyes.count/2..<possibleEyes.count]
leftEyes.sort { $0.size > $1.size }
rightEyes.sort { $0.size > $1.size }
leftEyes = leftEyes.dropLast(Int(Double(leftEyes.count) * 0.01))
rightEyes = rightEyes.dropLast(Int(Double(leftEyes.count) * 0.01))
let sumXleft = ( leftEyes.reduce(0) { $0 + $1.coord.x} ) / leftEyes.count
let sumYleft = ( leftEyes.reduce(0) { $0 + $1.coord.y} ) / leftEyes.count
let sumXright = ( rightEyes.reduce(0) { $0 + $1.coord.x} ) / rightEyes.count
let sumYright = ( rightEyes.reduce(0) { $0 + $1.coord.y} ) / rightEyes.count
let maskOriginalWidth = CGFloat(exactly: sizeX)!
let maskOriginalHeight = CGFloat(exactly: sizeY)!
let maskOriginalLeftEyePossitionX = CGFloat(exactly: sumXleft)!
let maskOriginalLeftEyePossitionY = CGFloat(exactly: sumYleft)!
let maskOriginalEyeDistance = CGPointDistance(from: CGPoint(x: sumXright, y: sumYright), to: CGPoint(x: sumXleft, y: sumYleft))
//This code assumes the image AND face orientation is always matching the same orientation!
//The code needs to be adjusted for other cases using UIImage.Orientation to get the orientation and adjusts the coordinates accordingly.
//CIDetector might also not detect faces which don't have the same orientation as the photo. Try to use CIDetectorImageOrientation to look for other orientations of no face has been detected.
//Also you might want to use other orientation points and scale values (right eye, nose etc.) in case the left eye, and left to right eye distance is not available.
//Also this code is very wordy, pretty sure it can be cut down to half the size and made simpler on many places.
let uiImageFace = UIImage(named: "photo3.jpg")
let ciImageFace = CIImage(image: uiImageFace!)
let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
let faces = faceDetector.features(in: ciImageFace!)
if let face = faces.first as? CIFaceFeature {
/*
Getting the distances and angle based on the original photo
*/
let faceAngle = face.faceAngle
let rotationAngle = CGFloat(faceAngle * .pi / 180.0)
//The distance in between the eyes of the original photo.
let originalFaceEyeDistance = CGPointDistance(from: face.leftEyePosition, to: face.rightEyePosition)
/*
Adjusting the mask and its eye coordinates to fit the original photo.
*/
//Setting the scale mask : original.
let eyeDistanceScale = maskOriginalEyeDistance / originalFaceEyeDistance
//The new dimensions of the mask.
let newMaskWidth = maskOriginalWidth/eyeDistanceScale
let newMaskHeight = maskOriginalHeight/eyeDistanceScale
//The new mask coordinates of the left eye in relation to the original photo.
let newMaskLeftEyePossitionX = maskOriginalLeftEyePossitionX / eyeDistanceScale
let newMaskLeftEyePossitionY = maskOriginalLeftEyePossitionY / eyeDistanceScale
/*
Adjusting the size values to fit the desired final size on the screen.
*/
//Using the width of the screen to calculate the new scale.
let screenScale = uiImageFace!.size.width / view.frame.width
//The new final dimensions of the mask
let scaledToScreenMaskWidth = newMaskWidth / screenScale
let scaledToScreenMaskHeight = newMaskHeight / screenScale
//The new final dimensions of the photo.
let scaledToScreenPhotoHeight = uiImageFace!.size.height / screenScale
let scaledToScreenPhotoWidth = uiImageFace!.size.width / screenScale
//The new eye coordinates of the photo.
let scaledToScreenLeftEyeFacePositionX = face.leftEyePosition.x / screenScale
let scaledToScreenLeftEyeFacePositionY = (uiImageFace!.size.height - face.leftEyePosition.y) / screenScale //reversing the y direction
//The new eye to corner distance of the mask
let scaledToScreenMaskLeftEyeX = newMaskLeftEyePossitionX / screenScale
let scaledToScreenMaskLeftEyeY = newMaskLeftEyePossitionY / screenScale
//The final coordinates for the mask
let adjustedMaskLeftEyeX = scaledToScreenLeftEyeFacePositionX - scaledToScreenMaskLeftEyeX
let adjustedMaskLeftEyeY = scaledToScreenLeftEyeFacePositionY - scaledToScreenMaskLeftEyeY
/*
Showing the image on the screen.
*/
let baseImageView = UIImageView(image: uiImageFace!)
//If x and y is not 0, the mask x and y need to be adjusted too.
baseImageView.frame = CGRect(x: CGFloat(exactly: 0.0)!, y: CGFloat(exactly: 0.0)!, width: scaledToScreenPhotoWidth, height: scaledToScreenPhotoHeight)
view.addSubview(baseImageView)
let maskImageView = UIImageView(image: uiMaskImage!)
maskImageView.frame = CGRect(x: adjustedMaskLeftEyeX, y: adjustedMaskLeftEyeY, width: scaledToScreenMaskWidth, height: scaledToScreenMaskHeight)
maskImageView.transform = CGAffineTransform(rotationAngle: rotationAngle)
view.addSubview(maskImageView)
}
}
func CGPointDistanceSquared(from: CGPoint, to: CGPoint) -> CGFloat {
return (from.x - to.x) * (from.x - to.x) + (from.y - to.y) * (from.y - to.y)
}
func CGPointDistance(from: CGPoint, to: CGPoint) -> CGFloat {
return sqrt(CGPointDistanceSquared(from: from, to: to))
}
}