History
| Version |
Date |
Notable changes |
| Initial version |
July 8, 2024 |
|
| V2 |
July 11, 2024 |
After lots of discussion, V2 of the proposal is ready. The initial proposal and many of the discussions can be found in the Appendices. |
Goals
Below are the goals of the proposed refactor of the InvokeAI Stable Diffusion backend:
- Improve the readability of the Stable Diffusion denoising code.
- Make it easier for developers to add new features to the Stable Diffusion denoising logic.
- Future: Enable community node contributors to extend the Stable Diffusion denoising process.
References
This document is based on the following work:
Proposal
Summary
- There is a minimal core denoising loop with a set of ‘callback points’. ‘Extensions’ use these callback points to extend the behaviour of the core loop.
- Each denoising feature (e.g. IP-Adapter, ControlNet, etc.) is contained in a single extension class.
- Multiple extension classes are applied at runtime to achieve the desired denoising behaviour.
- In addition to implementing callbacks, extensions can optionally implement a context manager to patch the UNet model.