Paintify LoRA
Link to download page on CivitAi
This LoRA attempts to replicate paintings style.
Trigger words:
- Global: "paify, fine art parody" (mandatory)
- Baroque: "baroque"
- Impressionism: "impressionism"
- Realism: "realism"
- Rococo: "rococo"
- Romanticism: "romanticism"
Favorite weights: 0.7(anime models) - 1(realistic models)
Especially for anime characters it can be used at very low heights for an overall effect.
At 0.5 weight:
At 0.2 weight:
Please note that the "Romanticism" tag is baked in the Stable Diffusion models, results will greatly vary according to the checkpoint.
Steps, Hires steps, CFG: doesn't matter
Hires Upscaler: 4x-UltraSharp
Checkpoints: works way better with realistic checkpoints, especially DreamShaper, but is usable with all. If you have to use an anime checkpoint, I recommend semi-realistic ones such as DarkSun.
Impressionism style is the easiest to use with anime checkpoints.
Combining LoRAs: that's the main point, it is usable with realistic and anime character LoRAs.
Effects can be combined or used separately.
Technical data for the generation of this LoRA can be found at the bottom of this page.
Random pictures generated with Paintify
Technical aspect
This one was more than tricky to make! The key is using a low tagging, as you don't want the style to be tied to specific keywords. Not tagging at all (excepted trigger words) should have worked too.
Trained with OneTrainer on public domain paintings provided by the National Gallery of Art.
- 173 paintings tagged "Baroque"
- 133 paintings tagged "Impressionism"
- 87 paintings tagged "Realism" (Repeated 2x during the training)
- 70 paintings tagged "Rococo" (Repeated 2x during the training)
- 144 paintings tagged "Romanticism"
Steps
- Download images
- Resize them: I made a PHP script to resize images with a minimum size of 300 pixels and a maximum of 576, converts them to PNG format and adding a prefix for better classification.
See the Code - Download it
Dimensions are grouped by multiples of 8 to facilitate batch processing.
Will give a try with cropping too, as resizing can lead to severally low resolution.<?php
$dest_rep = 'to_crop/output/'; // Output directory
$basefiles = 'to_crop/input/'; // Input directory
$files = scandir($basefiles);
$count = 0;
$max_size = 576; // Max dimension
foreach($files AS $useless => $name)
{
if($name != '.' && $name != '..')
{
$source_file = $basefiles . $name;
$dest_file = $dest_rep . 'painting_romanticism_' . $count . '.png'; // Change 'painting_romanticism_' for the prefix you want
if(substr($name,-4) == '.png')
{
$img = @imagecreatefrompng($source_file);
}
elseif(substr($name,-4) == '.jpg' || substr($name,-5) == '.jpeg')
{
$img = imagecreatefromjpeg($source_file);
}
elseif(substr($name,-5) == '.webp')
{
$img = imagecreatefromwebp($source_file);
}
elseif(substr($name,-5) == '.avif')
{
$img = imagecreatefromavif($source_file);
}
$original_x = imagesx($img);
$original_y = imagesy($img);
// Optimal ratio
$ratio = ($original_x / $original_y) > 1 ? 169 : 916;
$new_x = floor($original_x / 8) * 8;
$new_y = floor($original_y / 8) * 8;
if($ratio == 169)
{
$true_ratio = $original_x / $original_y;
if($original_x > $max_size)
{
$ratio_crop = $original_x / $max_size;
$new_x = $max_size;
$new_y = round($original_y / $ratio_crop);
$new_y = floor($new_y / 8) * 8;
}
}
else
{
$true_ratio = $original_y / $original_x;
if($original_y > $max_size)
{
$ratio_crop = $original_y / $max_size;
$new_y = $max_size;
$new_x = round($original_x / $ratio_crop);
$new_x = floor($new_x / 8) * 8;
}
}
if($new_x > 300 && $new_y > 300)
{
$dst_file = imagecreatetruecolor($new_x, $new_y);
if(substr($name,-4) == '.png' || substr($name,-5) == '.webp')
{
imagecolortransparent($dst_file, imagecolorallocatealpha($dst_file, 0, 0, 0, 127));
imagealphablending($dst_file, false);
imagesavealpha($dst_file, true);
}
imagecopyresampled($dst_file, $img, 0, 0, 0, 0, $new_x, $new_y, $original_x, $original_y);
imagepng($dst_file, $dest_file);
}
}
}
?> - Automatic tagging with Kohya_SS, using WD14 and a treshold of 0.8, adding "paintify" + painting style at the beginning
Treshold is high because I want the style to be applied globally. - I Use OneTrainer for the training: download settings.
As this is a style LoRA, learning rate is slowed and steps are increased.