Task 1: Semantic Style Frame Creation
Objective: Generate a stylized keyframe that semantically aligns a reference style with a target subject.
Current Approach:
- Use Flux Kontext as primary solution
- Implement Claude agent to prompt for subject and style extraction
- Remix reference image into target subject while maintaining style consistency
Future Iterations:
- Fine-tune Flux Kontext with open weights for direct processing
- Explore OmniGen2 as alternative solution
- Current testing shows Flux Kontext as best performing option
Task 2: Semantic Image Alignment with Keyframe
Objective: Precisely edit the generated style frame to match the exact keyframe positioning and composition.
Current State:
- Existing image model provides decent baseline
- Issue: Gets overpowered when using ControlNet Union
Implementation Options:
- Style Guide Implementation: Single-shot semantic fitting without manual ControlNet selection
- Video Model Transition: Use GPT 4o video model to semantically align frames through transition analysis
Requirements: